Photogrammetry-based bending monitoring and load identification of steel truss structures

Abstract

The bending of steel truss structures is an important gauge for detecting, identifying, and evaluating potential issues with structural safety performance. The limitations and high cost of traditional monitoring methods make it challenging to carry out stable long-term monitoring. Therefore, this paper developed a displacement monitoring system for steel truss structures which fulfill the requirements of having low cost, high stability, and ease of operation. The system is based on the improved sub-pixel positioning technology, achieving precise positioning in unfavorable conditions such as long structure-camera distance, angle skew, and dim light. Then, this system was calibrated through field experiment and compared with other measurement systems. Finally, a load identification method was developed to identify discrepancies between the true load and the design load. This method uses optimization functions to identify the true load applied in the experiment, and the optimization parameter obtained by a genetic algorithm iteration is output as the optimal solution. The results suggest that the photogrammetric system performs well in practical engineering applications and can provide advantages including high precision, low cost, simple operation, etc. Results obtained by the load identification method agree well with measurements obtained from the actual structure, and can serve as a tool for evaluating the mechanical properties of similar structures. This method monitors potential risks of steel truss structures, and greatly improve the stability and safety of such structures.

Keywords

steel truss structure loading experiment photogrammetry bending deformation monitoring load identification

Introduction

Essential indicators for the health of steel truss structures includes structural down-warping and bending that may be observed during a safety inspection of the truss (Chen et al., 2021; Miyashita et al., 2017). Current methods for the safety inspection of steel truss structures mostly focus on the assessment of the bending deformation of a single member. Inspection methods can be divided into three approaches: (1) Manual inspection of down-warping and bending deformation can be performed with simple equipment such as total stations. However, this approach is prone to errors and is time-consuming, especially if used in the long term. Difficult viewing angle that complicates the operation of total stations can produce more errors. (2) Three-dimensional laser scanning equipment can scan the entire or part of the steel structure to form a point cloud model. Through post processing steps, the point cloud model of the member in question can be converted into an engineering model for use in design and analysis (Liu et al., 2019; Wei et al., 2021a; 2021b). However, due to the high cost of the equipment and complexity in data processing, this method is also not suitable for the long-term monitoring purposes. (3) Vision-based displacement measurement system (Xu and Brownjohn, 2018) can be used to extract displacements experienced by the structure. Through digital image correlation (DIC), the displacement is extracted from captured videos of the structure during service. Imaging has advantages such as low cost, high precision, and easy operation, and accordingly, more researchers have begun to use this method in recent years. For the purposes of monitoring the overall down-warping of the truss studied in this paper, method one is overly laborious and is sensitive to the state of the project site, and method two is challenging due to the considerable amount of data and difficulty of implementing a long-term monitoring workflow. However, method three can realize multi-point synchronous monitoring and can estimate the overall down-warping of the truss by analyzing the down-warping of key nodes. Method three serves as the basis of this manuscript.

Vision-based displacement measurement systems can generally be divided into two parts: dynamic vision (Lydon et al., 2019) and static photogrammetry (Shao et al., 2020). Such systems provide the benefit of many advantages, which can replace traditional contact displacement measurement systems and some non-contact systems (Xu and Brownjohn, 2018). Using DIC (Sundla et al., 2015), template matching, feature point extraction, sub-pixel interpolation (Liu et al., 2021), and other technologies, the accuracy of displacement measurement can be improved. Feng et al. (2015a, 2015b) adopted an advanced template matching algorithm called up-sampling cross-correlation and developed it into a software package for real-time displacement extraction from video images. Debella-Gilo and Kääb, (2011) achieved sub-pixel precision for normalized cross-correlation and evaluated the influence of pixel resolution on the accuracies of displacement measurement through image matching. Kromanis and Kripakaran, (2021) employ coordinate transformation techniques, which use identified features to compute structural displacements from images. Fukuda et al. (2013) developed an algorithm that estimates displacement by tracking features on the structure without the need for installing a target panel. A sub-pixel technique was also proposed to reduce measurement errors cost-effectively. Dong et al. (2019) developed a robust vision-based displacement measurement method that can overcome disturbances such as changes in lighting and the presence of fog to achieve high sensitivity at the subpixel level. Chu et al. (2019) proposed a digital image-based method for overall structural deformation monitoring by utilizing image perspective transformation and edge detection. Their method had an error of less than 5%. Non-contact vision methods have also been reported for simultaneous multi-point measurement of structural displacements using one camera. These methods can greatly save costs and person-hours, and are thus more suitable for long-term monitoring even in challenging sites (Feng and Feng, 2016; 2017; Lydon et al., 2018; Xu et al., 2018).

Most of the technologies available for vision-based structural health monitoring are mature and can be found in many commercial products (Sony et al., 2019). Examples include smartphones with cameras and high computational efficiency (Wang et al., 2018; Zhu et al., 2021), cheap, high-resolution cameras (Fukuda et al., 2010), crewless aerial vehicles (Fryskowska et al., 2016), and robot sensors (Kromanis and Forbes, 2019), all of which represent a new era of intelligent monitoring systems for civil infrastructure (Dong et al., 2020; Feng and Feng, 2016). An issue that has attracted increasing attention is the accurate measurement of displacement in civil infrastructure such as bridges (Feng et al., 2015a; 2015b; Lee and Shinozuka, 2006; Lee et al., 2017; Ribeiro et al., 2014; Santos et al., 2012; Xu et al., 2019; Yu et al., 2020; Zhao et al., 2021), Very Long Baseline Interferometry (VLBI) antennas (Hyukgil et al., 2017), frame structures (Afrouz et al., 2019), and wind turbine rotor blades (Tesauro et al., 2014).

However, most of the research into displacement measurement, while applicable to a wide variety of structures, have not focused on steel truss structures. Still, rare identification and analysis of the actual load on the truss, and thus the finite element model correction method can be used to identify existing loads on a truss. Modified and optimized finite element modeling has become an indispensable tool for evaluating the bearing capacity and safety of structures. A key point of safety evaluation for steel truss structures is obtaining the structural response (i.e., output) to a certain load (i.e., input). An important aspect of bridge safety assessment is the measurement of input and output data. A challenge is that the input data can include undesired sources such as loads form construction activity. Liu et al. (2019) proposed a new method of model refinement and structural assessment for crooked tubular members of steel grid roofs. Their study represented the first report of using 3D imaging for inspecting crooked members. Their method inventively offers a new direction for refined assessment of existing grid structures with crooked members and is highly amenable to practical implementation. Wei et al. (2021a) proposed a new deformation inspection and monitoring method of grid structures using image-based 3D reconstruction. Their method is the first to automatically recognize and extract shape deformation of a structural member. Then Wei et al. (2021b) investigated a model for estimating the residual load-bearing capacity of an existing pin-pointed grid structure with crooked members. They described a new model updating method using static measurement data to obtain a practical numerical structure model. However, their approach hinged on several assumptions. The above methods facilitate the structural inspection and analysis, but an unmet challenge is to be able to assess whether observed large deformation (i.e., damage) was due to stiffness degradation or due to a large load. Therefore, identifying the actual input load that can cause the current deformation of the structure becomes the key to outputting the accurate response of the structure.

Therefore, to realize the deformation monitoring and load identification for steel truss structures, the following three-part framework is proposed (Figure 1):

(1) A sub-pixel positioning algorithm was developed for overall down-warping monitoring of steel truss structures. Theoretically, the algorithm can accurately identify the center of the target on the circular tube member under various adverse factors.

(2) A photogrammetry system was designed using the developed sub-pixel positioning algorithm. Data from field experiments and other measurement systems verify the accuracy and practicality of the system.

(3) A load identification method that leverages photogrammetry was developed to effectively identify the loads experienced by the truss. At the same time, the accuracy of this method was verified by vibration data. This method can predict the load on the structure when it deforms and can be used to evaluate the mechanical performance of the structure.

Figure 1.

Methodology framework.

Photogrammetric algorithm

Sub-pixel positioning

Sub-pixel interpolation

Edge detection in digital images is essential for advanced image processing tasks such as image segmentation, target recognition, region shape extraction, etc. The accuracy of traditional edge detection methods can only reach the single pixel level, which can no longer meet the needs of high-precision measurement. Thus, it is necessary to improve the accuracy to the sub-pixel level. At the sub-pixel level, the basic unit of the pixel is subdivided to improve the image resolution. However, sub-pixel edge points are often located in regions of the image that changes excessively; thus, this paper uses a bilinear interpolation algorithm to obtain the sub-pixel position of edge points.

Bilinear interpolation is a linear interpolation with two variables. The core idea is to perform linear interpolation in two directions. As shown in Figure 2, given Q₁₂, Q₂₂, Q₁₁, and Q₂₁, the point to be interpolated is P. The first step is to interpolate R₁ and R₂ in the x-axis direction, and then interpolate P according to R₁ and R₂.

Figure 2.

Bilinear interpolation.

If the image is grayscale, the value of P is:

\begin{array}{l} f_{P} (x, y) \approx \frac{f (Q_{11})}{(x_{2} - x_{1}) (y_{2} - y_{1})} (x_{2} - x) (y_{2} - y) + \frac{f (Q_{21})}{(x_{2} - x_{1}) (y_{2} - y_{1})} (x - x_{1}) (y_{2} - y) \\ + \frac{f (Q_{12})}{(x_{2} - x_{1}) (y_{2} - y_{1})} (x_{2} - x) (y - y_{1}) + \frac{f (Q_{22})}{(x_{2} - x_{1}) (y_{2} - y_{1})} (x - x_{1}) (y - y_{1}) \end{array}

(1)

If a coordinate system is selected so that the coordinates of the four known points of f_P are (0, 0), (0, 1), (1, 0), and (1, 1), the interpolation formula can be simplified as:

f_{P} (x, y) \approx f (0,0) (1 - x) (1 - y) + f (1,0) x (1 - y) + f (0,1) (1 - x) y + f (1,1) x y

(2)

The formula can be expressed in terms of matrices:

f_{P} (x, y) \approx [\begin{array}{c} 1 - x & x \end{array}] [\begin{array}{c} f (0,0) & f (0,1) \\ f (1,0) & f (1,1) \end{array}] [\begin{array}{c} 1 - y \\ y \end{array}]

(3)

The gray pixel boundary of the target can be subdivided as much as possible using sub-pixel interpolation, and each single gray pixel can be subdivided into a geometric multiple of gray sub-pixels of different shades. The black and white sub-pixel boundary of the target can be distinguished by combining the binarized image with the gray scale pixels. Finally, the method is used to extract the black and white sub-pixel boundary to obtain the accurate target edge, which has higher accuracy than a border extracted directly at the pixel level (Figure 3).

Figure 3.

Edge detection.

Circle fit

After obtaining the fine sub-pixel edge of a circular target via sub-pixel interpolation and edge detection, the circle fitting method can be used to obtain accurate coordinates for the center of the circle. In this paper, least square fitting serves the purpose of fitting the circle. The goal is to find an implicit equation that can fit the circle, and then substitute the coordinates of the border points into the implicit equation to obtain the distance from any point to geometric features. The fitting criteria is that the sum of the absolute values of the distance from the data points to the fitted circle (i.e., error) is minimum. Implicit equation (4) express the details of the geometry of the fitted circle through parameter vectors (5).

F (X, a) = 0

(4)

a = (a_{1}, \dots, a_{q})

(5)

Using the MATLAB circle fitting toolbox, the precise sub-pixel coordinates of the center of a circular target can be calculated. The edge of the circular target consists of black and white sub-pixel edges of the target obtained in Section 2.1.1. To reduce the error, the above method for solving the center coordinates and circle radius is used to optimize the fit of the circle, and the center coordinate with the smallest error can be obtained, as shown in Figure 4.

Figure 4.

Optimized circular fit.

MATLAB camera calibrator

To convert the pixel spacing between two sub-pixel coordinates into the actual distance, the camera needs to be calibrated. In order to obtain the 3D spatial position of a point on the surface of an object based on its corresponding point in the image, a geometric model of the camera needs to be established. The parameters of the model are derived from the camera parameters, which are often obtained through calibration experiments. Camera calibration has always been an essential part of photogrammetry. Self-calibration is a vital routine in photogrammetry triangulation, especially in high-precision close-range measurements (Remondino and Fraser, 2006).

Firstly, the method of using a planar target array for camera calibration was improved (Remondino and Fraser, 2006). The algorithm can calibrate the target on the surface of the circular tube in the photo, obtain a virtual planar target, and restore important parameters of the target. Then the MATLAB camera calibrator was used for camera calibration. The calibrator uses Zhang’s method (Tsai, 1987), which not only calibrates camera parameters but also give the pixel distance between the corners of the target panel chessboard surface. According to the pixel distance and the actual distance of the chessboard, an accurate mapping between pixel and true distance can be obtained, as shown in Figure 5.

Figure 5.

MATLAB camera calibrator.

Sub-pixel positioning algorithm

Sub-pixel positioning techniques are used here to develop an algorithm for displacement measurement. Most similar algorithms reported in literature are more suitable for the ideal laboratory environment. It is difficult to deal with the errors caused by distance, angle deviation, light intensity, and the surface of circular tubes encountered in field experiment. Therefore, the algorithm needs to be optimized and improved to minimize the impact of adverse factors.

Combining the theory in section 2.1, the improved algorithm can convert non-planar targets into virtual planar targets. In the process of photo binarization and sub-pixel interpolation, the algorithm can perform black and white correction on the target, achieving an ideal effect similar. On this basis, precision conversion and circle fitting can be carried out to achieve accurate processing of the target.

To test the accuracy of the improved algorithm, a checkerboard target is needed. As shown in Figure 6(a), a target with 11 × 7 checkers with the side length of 10 × 10 mm was used for identification. Three white circles A, B, and C served as an additional target. The AB center to center distance is 100 mm, and for AC and BC, the center-to-center distances are 108 mm and 107 mm, respectively. A 13 MP smartphone camera was used to take photos, Figure 6(b)–(d) shows the three photos taken by the smartphone at different distances, light intensities, and angle deviations. Because of the large number of target pixels in the picture, the algorithm runs can exceed available computer memory. Thus, accuracy should be less than 1 mm using a 10x interpolation test. The results of algorithm processing are shown in Table 1.

Figure 6.

Target and photos of algorithm test. (a) Target, (b) Photo 1, (c) Photo 2, (d) Photo 3.

Table 1.

Results of algorithm test.

	Camera calibration (mm/Pixel)	Distance (mm) and error	AB	AC	BC
Actual	/	Distance	100	108	107
Photo 1	0.253	Distance	100.645	108.407	107.217
Photo 1	0.253	Error	0.65%	0.38%	0.20%
Photo 2	0.102	Distance	99.159	105.967	104.678
Photo 2	0.102	Error	0.84%	1.89%	2.18%
Photo 3	0.088	Distance	99.915	106.914	105.505
Photo 3	0.088	Error	0.09%	1.01%	1.40%

It can be seen from Table 1 that when the photograph distance is close, the algorithm has a high sensitivity to the photograph angle and light intensity, and the accuracy is also the lowest. On the other hand, when the photograph distance is far, the angle and light intensity still impacts accuracy, but the impact is damped due to the long viewing distance and large field of view. A suitable distance thus helps the accuracy of the algorithm, up to reaching the theoretically possible accuracy. Overall, the maximum error is only 2.18%, showing that the improved algorithm has suitable for field deployment.

Field experiment

Experimental setup

Experimental site

This section examines the down-warping and deformation of the truss bottom chord members in the truss roof of an exhibition hall that was under construction. Due to the on-site environmental limitations and difficulty of installing equipment, the first truss near the curtain wall was selected for the elastic loading experiment, as shown in Figure 7(a). To ensure safety, the maximum loading condition stayed below 2000 kg for each of the five nodes in the midspan. Load cases were divided per 500 kg. Loads were applied by filling a bucket with water and thus the weights of the bucket, hanging rope, tray, and other loads, which total 200 kg also need to be considered (Figure 7(b)).

Figure 7.

Experimental site. (a) Experimental truss, (b) Loading device.

As shown in Figure 8, points A ∼ E are the loading points, points A ∼ C are the target monitor points, and the targets are pasted on the members. The target design and layout are shown in Figure 9.

Figure 8.

Truss points.

Figure 9.

Photogrammetry. (a) Target design, (b) Target layout.

Static analysis

The MIDAS finite element software was used to establish the finite element model of the exhibition hall truss, as shown in Figure 10(a). The local finite element model of the target truss is shown in Figure 10(b). All members are built by beam elements with Q345B material.

Figure 10.

Finite element models. (a) The whole model of truss roof, (b) Target truss.

The bending deformation and stress distribution of the truss can be obtained through static analysis, as shown in Figure 11 and Table 2. For the sake of conciseness, the diagrams of each load condition are not shown here. In Figure 11(a), the red color denotes the maximum vertical displacement while the blue color represents minimum vertical displacement. Likewise, in Figure 11(b), red is the maximum tensile stress and blue is the maximum compressive stress. Table 3 shows the relative displacement of the target monitor points under various load conditions.

Figure 11.

Simulation results (Condition 1). (a) Bending deformation, (b) Stress distribution.

Table 2.

Simulation results.

Condition	Load	Maximum vertical displacement (mm)	Maximum tensile stress (MPa)	Maximum compressive stress (MPa)
1	Self-weight	−65.338	125.163	89.702
2	Empty bucket	−65.865	126.44	90.374
3	500 L water	−67.182	129.634	92.055
4	1000 L water	−68.499	132.829	93.736
5	1500 L water	−69.816	136.025	95.418
6	2000 L water	−71.134	139.221	97.100

Table 3.

Relative displacement of points under various conditions.

Relative displacement (mm)	Point A	Point B	Point C
Condition 2–1	−0.382	−0.477	−0.512
Condition 3–2	−0.955	−1.193	−1.280
Condition 4–3	−0.956	−1.194	−1.281
Condition 5–4	−0.956	−1.194	−1.281
Condition 6–5	−0.956	−1.195	−1.282

The simulation results indicate that the stress of the members stay within the limits of material strength and the range of observed stresses is narrow. The members vertically displace by more than 1 mm under the main loading conditions. The simulation results show that the experimental loading scheme is feasible.

Dynamic characteristics

The model and load conditions in Section 3.1.2 were used to analyze the first two dominant mode shapes of the target truss under different loading conditions (Figure 12). The first mode shape is vertical bending and the second mode shape is transverse torsion. The frequency of the various loading conditions as shown in Table 4.

Figure 12.

First two mode shapes of the target truss. (a) Vertical bending, (b) Transverse torsion.

Table 4.

Vibration shape and frequency.

Vibration shape	Vertical bending (Hz)	Transverse torsion (Hz)
Condition 1	2.671	5.594
Condition 2	2.664	5.583
Condition 3	2.644	5.554
Condition 4	2.619	5.521
Condition 5	2.590	5.486
Condition 6	2.560	5.449

Experimental progress

The field experiment started at 17:00 on September 17, 2021, and ended at 17:00 on September 18, 2021, with specific load scheduling listed in Table 5. Note that the only photogrammetry data was captured for the self-weight condition due to data corruption in other equipment caused by unexpected movements, power failure, and other disturbances.

Table 5.

Experimental load schedule.

Condition	Load	Time	Notes
1	Self-weight	15:00 ∼ 16:00 on the 17th	Movement of other measurement systems
2	Empty bucket	9:15 ∼ 9:30 on the 18th	/
3	500 L water	10:30 ∼ 11:00 on the 18th	/
4	1000 L water	14:15 ∼ 14:30 on the 18th	Power failure of Doppler meter
5	1500 L water	14:55 ∼ 15:20 on the 18th	/
6	2000 L water	15:35 ∼ 15:55 on the 18th	/

Photogrammetric system

The displacement calibration of the photogrammetry requires paper targets to be displayed on the members near points A ∼ C. The targets will be tracked by three cameras (Table 6), which were connected to a chassis containing computers, routers, switches, and other equipment installed on the curtain wall (Figure 13(a)). The distance between the camera and the truss member was approximately 8 m. The three cameras monitor the member near points A ∼ C, and example photographs shown in Figure 13(b)–(d).

Table 6.

Camera parameters.

Camera parameters
Type	MV-CE200-10GC
Name	20 million pixel mesh interface area array camera, IMX 183, color
Pixel size	2.4 µm × 2.4 µm
Target size	1”
Resolution ratio	5472 × 3648
Lens parameters
Type	MVL-KF2528M-12MPE
Name focal distance	25 mm, F2.8, 1.1″, 12 million resolution ratio, C-interface lens 25 mm

Figure 13.

Photogrammetry system. (a) Chassis and camera (Point A), (b) Point A, (c) Point B, (d) Point C.

In this paper, the distance between the cameras and the corresponding truss nodes is about 8 m, and the view field of each camera is about 4.2 * 2.8 m, with a pixel accuracy pre-estimated at 0.77 mm/pixel. Based on the number of targets in the view field, it is rough to determine the number of targets that can be measured simultaneously. In Figure 13(b)–(d), excluding scaffolding obstruction, each camera can simultaneously measure eight targets. The relative displacement variation curve formed by these targets can obtain the precise displacement of the truss nodes.

In the field experiment, wait for the water injection to stop and load facilities such as cables and buckets to stop shaking, then start collecting data. Take 10 photos of each condition, with a sampling frequency of 1 min per photo. After the field experiment, the photogrammetric system entered the long-term monitoring stage, automatically taking 24 photos every day with a sampling frequency of 1 h per photo, and removing photos under no light conditions.

Regarding the data processing equipment, the processor is AMD Ryzen 7 5700 G with Radeon Graphics 3.80 GHz, the system is Microsoft Windows 10 Professional Edition, and the software is MATLAB, 2017a. Due to the use of high-definition cameras, the photo storage capacity reaches 57.1 MB, resulting in the algorithm processing speed of approximately 1 min per photo.

At the completion of the experimental setup, remote control and long-term monitoring were enabled. Compared with traditional monitoring methods, the cost and workload are significantly reduced.

Other measurement systems

To verify the accuracy of the photogrammetry system, a 3D scanner and a laser doppler meter were used to measure the displacement, and a vibration transducer was used to measure the vibration during the elastic loading experiment.

3D scanner

The Trimble TX8 laser 3D scanner was used to obtain the point cloud model of the truss, and the equipment parameters are listed in Table 7. The whole truss can be scanned, and a photo of the deployed scanner is shown in Figure 14.

Table 7.

Equipment parameters.

Name	Trimble TX8 laser 3D scanner
Scan principle	Vertical rotating mirror on horizontal rotating foundation
Range principle	Tianbao lightning technology combining phase and pulse
Scan speed	1 million points/second
Range measurement	0.6–120 m
Accuracy	<1 mm
Field of view	360° × 317°
Scan time	10 min
Point spacing at 30 m	5.7 mm
Number of points	550 million

Figure 14.

Trimble TX8 laser 3D scanner.

The advantage of the scanner is that it can be quickly setup to automatically scan and obtain the 3D point cloud data of the whole truss. However, the subsequent massive data processing can be very complex, and the measurements are highly sensitive to disturbances from the environment. The high cost is also not conducive to long-term monitoring.

Single-point laser doppler meter

A single-point laser doppler meter was used to accurately measure the down-warping of the midspan node. The specific parameters are listed in Table 8, and the meter is shown in Figure 15.

Table 8.

Equipment parameters.

Name	Single-point laser Doppler meter
Maximum frequency	24 MHz
Maximum displacement	±1.225 m
Accuracy	<0.1 mm
Focus mode	Manual focusing
Focal distance of objective lens	100 mm
Work distance	0.5–100 m
Minimum spot diameter	53 μm

Figure 15.

Single-point laser doppler meter.

The doppler meter has the highest accuracy and is most suitable for verification tasks. The disadvantage is that it can only measure a single point. Other advantages and disadvantages are similar to the 3D scanner, a single point time history dataset as complex as the 3D point cloud.

941B vibration transducer

During the elastic loading experiment, the vibration of the truss was measured using 941B vibration transducers to analyze the dynamic characteristics of the truss and to help identify loads. Environmentally transmitted excitations, including from the construction work on site were the source of vibrations. One vertical and one horizontal vibration transducer are installed at points A ∼ C, as shown in Figure 16 and Table 9.

Figure 16.

Photo of equipment and installation.

Table 9.

Equipment and installation.

Equipment number	1st gear sensitivity (Vs²/m)	Installation position
V19447	0.333585	Point A vertical
H19347	0.315435	Point A transverse
V19448	0.314698	Point B vertical
H19348	0.255334	Point B transverse
V19449	0.329134	Point B vertical
H19349	0.312641	Point B transverse

Data is collected for every condition, and the data acquisition instrument and field monitoring effect are shown in Figure 17.

Figure 17.

Experimental site.

Experimental results

Experimental data

Photogrammetry

Due to the small number of single target pixels, 100x interpolation can be used to improve the accuracy to within 0.1 mm. According to the optimized sub-pixel positioning algorithm, the relative pixel distance and actual distance of each target under each condition can be obtained. Fitting the data can reveal the deformation curve of the member near the targets. The relative displacement of the targets between each condition is shown in Table 10.

Table 10.

Relative displacements of points A ∼ C between the different conditions.

Relative displacement (mm)	Point A	Point B	Point C
Condition 2–1	−0.400	−0.495	−0.527
Condition 3–2	−0.956	−1.155	−1.222
Condition 4–3	−0.990	−1.222	−1.298
Condition 5–4	−1.033	−1.214	−1.274
Condition 6–5	−1.050	−1.263	−1.336

3D scanner

The cylindrical axis fitting algorithm in MATLAB was used to extract the central axis of the point cloud cylinder component and obtain the relative displacement of the target points under different conditions, as shown in Table 11.

Table 11.

Relative displacement.

Relative displacement (mm)	Point A	Point B	Point C
Condition 2–1	/	/	/
Condition 3–2	−0.934	−1.198	−1.286
Condition 4–3	−1.278	−1.338	−1.359
Condition 5–4	−0.870	−1.052	−1.112
Condition 6–5	−0.981	−1.309	−1.419

Note: no data was collected under condition 1.

Doppler meter

After calibrating the data collected by the doppler meter, the time domain waveform and time-frequency diagram of the midspan point can be obtained. The relative displacement between each condition of the measured point obtained by the conversion is listed in Table 12.

Table 12.

Relative displacement between conditions at point C.

Relative displacement (mm)	Point C
Condition 2–1	/
Condition 3–2	−1.210
Condition 4–3	−0.020*
Condition 5–4	−0.050*
Condition 6–5	−1.320

Note: no data was collected in condition 1, and the data was invalid due to power failure between condition 4–3 and condition 5–4.

941B vibration transducer

The spectral data of the vibration measurements were obtained from each vibration transducer at each point and for each condition. Table 13 lists the frequencies of the first two mode shapes under each condition.

Table 13.

Frequency.

Frequency (Hz)	Vertical bending	Transverse torsion
Condition 1	2.691	5.585
Condition 2	2.662	5.551
Condition 3	2.643	5.540
Condition 4	2.588	5.511
Condition 5	2.553	5.496
Condition 6	2.550	5.412

Analysis results

Model validation

The vibration measurement data (Table 13) are compared with the simulation data (Table 4) as shown in Table 14. Note that the error of the measured frequency is relative to the frequency obtained from simulation.

Table 14.

Relative error of the measured frequencies against the simulated.

Vibration shape	Vertical bending (%)	Transverse torsion (%)
Condition 1	0.74	0.16
Condition 2	0.08	0.58
Condition 3	0.04	0.25
Condition 4	1.20	0.18
Condition 5	1.45	0.18
Condition 6	0.39	0.68

The difference between the measured frequency of the truss and the simulation results is minimal, indicating that the model is consistent with reality. At this time, the structure has not experienced damage, and thus it is intuitive that the model is consistent with the measured data, which demonstrates the accuracy of the finite element model.

Displacement comparison

The characteristics of the three sets of displacement measurement equipment are summarized in Table 15. As seen in the table, the accuracy of photogrammetry is among the highest, and the cost is the lowest. At the same time, the difficulty of equipment operation and data processing is relatively simple, which is suitable for long-term monitoring.

Table 15.

Performance comparison.

Measurement system	Photogrammetry	3D scanner	Doppler meter	Total station
Accuracy	<0.1 mm	<1 mm	<0.1 mm	<1 mm
Cost	Low	High	High	High
Difficulty of equipment installation	Complex	Simple	Simple	Simple
Environmental restrictions	Less	More	More	More
Difficulty of equipment operation	Simple	Simple	Complex	Simple
Difficulty of data processing	Simple	Complex	Complex	Simple
Remote control	√	×	×	×
Suitable for long-term monitoring	√	×	×	×

Figure 18 summarizes the displacement data under each condition, including the results of the simulation model and the measurements from photogrammetry, 3D scanning, and the doppler instrument.

Figure 18.

Relative displacement. (a) Condition 2–1, (b) Condition 3–2, (c) Condition 4–3, (d) Condition 5–4, (e) Condition 6–5.

At the design load, most of the 3D scanning data had significant errors while the photogrammetry data had minimal error. The high 3D scanning error is likely related to the limitations of the equipment, and the error is exacerbated when the 3D scanning is carried out at a long distance. The photogrammetric data agrees well with the doppler data, which demonstrates the accuracy of photogrammetry; however, there is a difference between the measured data and the simulation data, likely because the actual load is not the same as the design load. The actual load needs to be corrected according to the measured displacement data and then verified with the vibration measurement data.

Long-term monitoring results

After the field experiment, the photogrammetric system entered the long-term monitoring stage. Due to frequent construction near the curtain wall by workers from September 19th to October 11th, resulting in vibrations and obstructions that made it difficult to use photogrammetric data, the long-term monitoring data was used from October 12th to November 30th. Based on the first photo taken before construction on the morning of October 12th, Figure 19 shows the long-term monitoring results of the three monitor points.

Figure 19.

Long-term monitoring results.

It can be seen that in long-term monitoring, the relative displacement of the three monitor points was always between −6 and +6 mm, and the overall change was relatively stable, with no obvious bending or deformation of the truss. After the construction near the monitoring truss was completed on October 30th, the relative displacement of the three monitoring points gradually returned to 0 mm, indicating that the truss still had good stability and safety during the construction phase.

Load identification

Identification method

A key point of safety evaluation for steel truss structures is obtaining the structural response (i.e., output) to a certain load (i.e., input). Liu et al. (2019) and Wei et al. (2021a, 2021b) used static measurement data to obtain a practical numerical structure model. Their methods facilitate the structural inspection and analysis, but an unmet challenge is to be able to assess whether observed large deformation (i.e. damage) was due to stiffness degradation or due to a large load. Therefore, identifying the actual input load that can cause the current deformation of the structure becomes the key to outputting the accurate response of the structure.

The load identification method proposed in this paper is mainly based on the measured structural deformation index. The method relies on an optimization function to identify the true load applied in the experiment. Different loads are iteratively applied to a simulated truss until the deformation of the simulated truss matched the real truss. The load was selected as the optimization parameter because the deformation of the truss is mainly caused by factors such as self-weight, roof load, and the load applied in the experiment. The load intensity was controlled by adjusting the water level in a container on the truss. The collective mass of the steel cable, the bucket, and the tray required for water storage is about 200 kg, and the marking of bucket water storage is relatively low resolution. Therefore, there is a non-negligible difference between the actual and design loads, and thus the experimental loads need to be optimized and corrected.

First, the load F of each loading condition is substituted into the finite element model in Section 3.2.2 to obtain the deformation index of the self-weight conditions of the three monitored points. Therefore, the deformation index X can be regarded as a function of F, and for the self-weight conditions, an objective function in the form of equation (6) can be constructed:

\min f (F) = \sqrt{\frac{\sum_{i = 3} {(X_{C} (i) - X_{M} (i))}^{2}}{3}}

(6)

where X_M(i) is the experimentally measured value of the relative self-weight condition of the ith monitored point, and X_C(i) is the calculated value of the relative self-weight condition of the ith monitor point.

After determining the correction parameter and the value of the objective function, it is necessary to develop an appropriate load identification algorithm. Here, a genetic algorithm is used. Since the container does not deform or otherwise change, the water injection amount under each working condition should be consistent. However, the volume markers of the container are not exact, and thus there is a deviation between the actual water injection amount and the expected, design amount. Since there is only one variable, the algorithm only needs to select the correct population number and genetic optimization times and then iterate to find the minimum value of equation (6).

Finally, when the accuracy requirements of equation (7) are satisfied, the optimization parameter obtained by the current iteration is output as the optimal solution.

Δ f = f_{k + 1} - f_{k} \leq 0.001

(7)

where K represents the number of iterations.

Figure 20 summarizes the process of load identification.

Figure 20.

Load identification process.

Load identification

Guided by the principles and methods discussed above, ANSYS and MATLAB were jointly used to develop a simulation optimization program. The parameters of the structural finite element model were modified based on the deformation index of the relative self-weight condition of the three monitored points. The program was developed primarily in MATLAB, and the optimization toolbox in MATLAB was used for the global search and iterative optimization process. The optimization parameters of each iteration are written into a file that can be read by ANSYS. The objective function value is calculated in MATLAB and during each iteration the MATLAB script calls ANSYS to perform numerical analysis and generates an output file. The load identification results of each loading condition are summarized in Table 16 and shown in Figure 21.

Table 16.

Load comparison.

Condition	Design load (N)	Optimization load (N)	Correction factor
1	Self-weight	Self-weight	/
2	2000	2072.6	1.0363
3	7000	6927.9	0.9897
4	12,000	12,043.2	1.0036
5	17,000	17,197.2	1.0116
6	22,000	22,437.8	1.0199

Figure 21.

Relative displacement. (a) Condition 2–1, (b) Condition 3–2, (c) Condition 4–3, (d) Condition 5–4, (e) Condition 6–5.

The relative displacement of the monitored points obtained from the modified load analysis agrees well with the photogrammetric measurements. However, this does not mean that the photogrammetric measurements are accurate. It is also necessary to analyze the structure's dynamic characteristics due to load modifications and compare the obtained frequencies with the vibration measurements to verify the accuracy of model correction.

Load verification

The load obtained from the load identification task in Section 4.2 was substituted into the finite element model from Section 3.1.2 to analyze the dynamic characteristics of the truss under various load conditions. The results are listed in Table 17.

Table 17.

Frequency.

Frequency (Hz)	Vertical bending	Transverse torsion
Condition 1	2.671	5.594
Condition 2	2.653	5.575
Condition 3	2.632	5.546
Condition 4	2.604	5.513
Condition 5	2.574	5.477
Condition 6	2.542	5.438

Table 18 compares Table 17 with Table 4 to obtain the relative error between the theoretical frequency and the measured frequency of the truss after load identification.

Table 18.

Relative error.

Vibration shape	Vertical bending (%)	Transverse torsion (%)
Condition 1	0.74	0.16
Condition 2	0.34	0.43
Condition 3	0.42	0.11
Condition 4	0.62	0.04
Condition 5	0.82	0.35
Condition 6	0.31	0.48

Further comparing Table 18 with Table 14 shows that the frequency changes minimally before and after load identification. The average error of the vibration frequency before load identification is 0.49% (minimum of 0.04% and maximum of 1.45%). Meanwhile, the average error after load identification is 0.40% (minimum of 0.04% and maximum of 0.82%). Therefore, the photogrammetric data can be considered accurate and reliable.

Conclusion

In this paper, a photogrammetry-based method for monitoring bending deformation and load identification for a steel truss structure was developed and demonstrated. The following conclusions can be drawn.

(1) The optimized sub-pixel location algorithm presented in this paper is highly sensitive to viewing angle and light intensity when the shooting distance is short. Conversely, a suitable shooting distance reduces the influence of viewing angle and lighting and yields higher accuracy. Overall, the maximum error is only 2.18% even at close distances, and the test results demonstrated that the improved algorithm is practical, especially in cases where there are longer distances between the monitored structure and the camera.

(2) Displacement measurements obtained from field tests show that 3D scanning suffers from significant errors due to limitations of the 3D scanning equipment. The error becomes more severe at longer distances. On the other hand, photogrammetric measurements agreed well with the doppler measurements, which served as the gold standard in this paper. While this agreement verifies the accuracy and reliability of photogrammetric measurements, there are still differences between the measured versus the theoretical displacement. This discrepancy may have been caused by the actual loads not being exactly equal to the design load.

(3) An ANSYS-MATLAB simulation optimization program utilized the measured structural deformation index to correct for the difference between true load and the design load (i.e., load optimization). After optimization, the load analysis and the dynamic characteristics of the truss agreed well with the photogrammetric displacement measurements, thus verifying the accuracy of the program and the photogrammetric data. This method can also be used to identify the actual loads on the truss during the monitoring period and provide guidance for the safety assessment of the truss.

(4) The calibration and verification results of the photogrammetry system show that the accuracy of displacement measurement by the system is less than the theoretical accuracy of prediction. Compared with the 3D scanner and doppler meter, the photogrammetry system has the advantages of low equipment cost, simple equipment operation, simple data processing, and suitability for long-term monitoring. The calibration experiment demonstrated that photogrammetry has a good potential for monitoring the bending deformation of steel truss structures.

(5) The photogrammetric system and load identification method verified by the study can effectively be used for long term monitoring of steel truss structure during the construction and operation stage. The long-term monitoring results also show that the truss has good stability and safety in the construction phase, which helps to reduce the possible risks of steel truss structure. The method may be able to guide the technical development of steel truss structures.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (Grant No. 52192662). The authors express their sincere appreciation to their supports.

ORCID iD

Yu-Fei Liu

References

Afrouz

Razavi

Pourkand

, et al. (2019) Dynamic displacement of an aluminum frame using close range photogrammetry. Buildings 9(8): 176. DOI: 10.3390/buildings9080176.

Chen

Yuan

Wang

(2021) Reliability fuzzy comprehensive evaluation based on construction monitoring long-span arch truss. E3S Web of Conferences 293: 02035. DOI: 10.1051/e3sconf/202129302035.

Chu

Zhou

Deng

, et al. (2019) An overall deformation monitoring method of structure based on tracking deformation contour. Applied Sciences 9(21): 4532. DOI: 10.3390/app9214532.

Debella-Gilo

Kääb

(2011) Sub-pixel precision image matching for measuring surface displacements on mass movements using normalized cross-correlation. Remote Sensing of Environment 115(1): 130–142. DOI: 10.1016/j.rse.2010.08.012.

Dong

Celik

Catbas

, et al. (2019) A robust vision-based method for displacement measurement under adverse environmental factors using Spatio-Temporal context learning and Taylor approximation. Sensors 19(14): 3197. DOI: 10.3390/s19143197.

Dong

Celik

Catbas

, et al. (2020) Structural displacement monitoring using deep learning-based full field optical flow methods. Structure and Infrastructure Engineering 16(1): 51–71. DOI: 10.1080/15732479.2019.1650078.

Feng

(2016) Vision-based multipoint displacement measurement for structural health monitoring. Structural Control and Health Monitoring 23(5): 876–890. DOI: 10.1002/stc.1819.

Feng

(2017) Experimental validation of cost-effective vision-based structural health monitoring. Mechanical Systems and Signal Processing 88: 199–211. DOI: 10.1016/j.ymssp.2016.11.021.

Feng

Ozer

, et al. (2015a) A vision-based sensor for noncontact structural displacement measurement. Sensors 15(7): 16557–16575. DOI: 10.3390/s150716557.

10.

Feng

Fukuda

Feng

, et al. (2015b) Nontarget vision sensor for remote measurement of bridge dynamic response. Journal of Bridge Engineering 20(12): 04015023. DOI: 10.1061/(ASCE)BE.1943-5592.0000747.

11.

Fryskowska

Kedzierski

Grochala

, et al. (2016) Calibration of low cost RGB and NIR UAV cameras. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 41(B1): 817–821. DOI: 10.5194/isprsarchives-XLI-B1-817-2016.

12.

Fukuda

Feng

Shinozuka

(2010) Cost-effective vision-based system for monitoring dynamic response of civil engineering structures. Structural Control and Health Monitoring 17(8): 918–936. DOI: 10.1002/stc.360.

13.

Fukuda

Feng

Narita

, et al. (2013) Vision-based displacement sensor for monitoring dynamic response using robust object search algorithm. IEEE Sensors Journal 13(12): 4725–4732. DOI: 10.1109/JSEN.2013.2273309.

14.

Hyukgil

Yun

Jinsang

, et al. (2017) A static displacement monitoring system for VLBI antenna using close-range photogrammetry. Applied Sciences 7(11): 1125. DOI: 10.3390/app7111125.

15.

Kromanis

Forbes

(2019) A low-cost robotic camera system for accurate collection of structural response. Inventions 4(3): 47. DOI: 10.3390/inventions4030047.

16.

Kromanis

Kripakaran

(2021) A multiple camera position approach for accurate displacement measurement using computer vision. Journal of Civil Structural Health Monitoring 11(3): 661–678. DOI: 10.1007/s13349-021-00473-0.

17.

Lee

Shinozuka

(2006) A vision-based system for remote sensing of bridge displacement. NDT & E International 39(5): 425–431. DOI: 10.1016/j.ndteint.2005.12.003.

18.

Lee

Cho

, et al. (2017) Computer vision-based structural displacement measurement robust to light-induced image degradation for in-service bridges. Sensors 17(10): 2317. DOI: 10.3390/s17102317.

19.

Liu

Fan

, et al. (2019) Refined safety assessment of steel grid structures with crooked tubular members. Automation in Construction 99: 249–264. DOI: 10.1016/j.autcon.2018.11.027.

20.

Liu

Nie

, et al. (2021) Enhancement of contour smoothness by substitution of interpolated sub-pixel points for edge pixels. IEEE Access 9: 44236–44246. DOI: 10.1109/ACCESS.2021.3061533.

21.

Lydon

del Rincon

, et al. (2018) Development and field testing of a time-synchronized system for multi-point displacement calculation using low-cost wireless vision-based sensors. IEEE Sensors Journal 18(23): 9744–9754. DOI: 10.1109/JSEN.2018.2853646.

22.

Lydon

Taylor

, et al. (2019) Development and field testing of a vision-based displacement system using a low cost wireless action camera. Mechanical Systems and Signal Processing 121: 343–358. DOI: 10.1016/j.ymssp.2018.11.015.

23.

Miyashita

Iwasaki

Nagai

(2017) Performance evaluation of hinges in long-span cantilever steel truss bridges by structural health monitoring. Sensors and Materials 29(2): 179–186. DOI: 10.18494/SAM.2017.1419.

24.

Remondino

Fraser

(2006) Digital camera calibration methods: considerations and comparisons. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 36(5): 266–272. DOI: 10.3929/ethz-b-000158067.

25.

Ribeiro

Calçada

Ferreira

, et al. (2014) Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system. Engineering Structures 75: 164–180. DOI: 10.1016/j.engstruct.2014.04.051.

26.

Santos

Costa

Batista

(2012) Calibration methodology of a vision system for measuring the displacements of long‐deck suspension bridges. Structural Control and Health Monitoring 19(3): 385–404. DOI: 10.1002/stc.438.

27.

Shao

Zhou

Deng

, et al. (2020) Experiment of structural geometric morphology monitoring for bridges using holographic visual sensor. Sensors 20(4): 1187. DOI: 10.3390/s20041187.

28.

Sony

Laventure

Sadhu

(2019) A literature review of next-generation smart sensing technology in structural health monitoring. Structural Control and Health Monitoring 26(3): e2321. DOI: 10.1002/stc.2321.

29.

Sundla

Punning

Aabloo

, et al. (2015) Semi-automatic deflection measurement using digital image correlation. In: 2015 19th International Conference on System Theory, Control and Computing (ICSTCC), Cheile Gradistei, Romania, 14–16 October 2015, 325–330.

30.

Tesauro

Eder

Nielsen

(2014) Measurement of local relative displacements in large structures. The Journal of Strain Analysis for Engineering Design 49(5): 301–314. DOI: 10.1177/0309324713519622.

31.

Tsai

(1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal of Robotics and Automation 3(4): 323–344. DOI: 10.1109/jra.1987.1087109.

32.

Wang

Liu

, et al. (2018) Notice of removal: structural displacement monitoring using smartphone camera and digital image correlation. IEEE Sensors Journal 18(11): 4664–4672. DOI: 10.1109/JSEN.2018.2828139.

33.

Wei

Fan

Liu

, et al. (2021a) Automated inspection and monitoring of member deformation in grid structures. Computer-Aided Civil and Infrastructure Engineering 37(10): 1277–1297. DOI: 10.1111/mice.12766.

34.

Wei

Liu

Fan

, et al. (2021b) Safety assessment of existing pin-jointed grid structures with crooked members using static model updating. Engineering Structures 247: 113107. DOI: 10.1016/j.engstruct.2021.113107.

35.

Brownjohn

JMW

(2018) Review of machine-vision based methodologies for displacement measurement in civil structures. Journal of Civil Structural Health Monitoring 8(1): 91–110. DOI: 10.1007/s13349-017-0261-4.

36.

Brownjohn

JMW

Kong

(2018) A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge. Structural Control and Health Monitoring 25(5): e2155. DOI: 10.1002/stc.2155.

37.

Brownjohn

JMW

Huseynov

(2019) Accurate deformation monitoring on bridge structures using a cost-effective sensing system combined with a camera and accelerometers: case study. Journal of Bridge Engineering 24(1): 05018014. DOI: 10.1061/(ASCE)BE.1943-5592.0001330.

38.

Fang

Meng

, et al. (2020) Measurement of quasi-static and dynamic displacements of footbridges using the composite instrument of a smartstation and an accelerometer: case studies. Remote Sensing 12(16): 2635. DOI: 10.3390/rs12162635.

39.

Zhao

Deng

, et al. (2021) Trajectory tracking and load monitoring for moving vehicles on bridge based on axle position and dual camera vision. Remote Sensing 13(23): 4868. DOI: 10.3390/rs13234868.

40.

Zhu

Zhang

(2021) A marker-free method for structural dynamic displacement measurement based on optical flow. Structure and Infrastructure Engineering 18(1): 84–96. DOI: 10.1080/15732479.2020.1835999.