Vibration measurement in machine vision based on ROI gray-scale projection feature matching algorithm

Abstract

The current machine vision-based vibration measurement faces numerous challenges, such as low sampling frequency, excessive computational time, and expensive high-definition, high-frame-rate industrial cameras. In response to these issues, this study proposed a feature-matching algorithm that combines ROI image interpolation with gray-scale projection. This algorithm allows cost-effective industrial cameras with lower resolution but higher frame rates. The system effectively identifies modal parameters by executing interpolation and gray-scale projection processing on the ROI images and matching the generated gray-scale projection features. This allows for precisely capturing dynamic positional changes at measurement points. To validate the effectiveness of the proposed method, this study performed impact tests on a laboratory-based model of a beam that was simply supported. The experiments aimed to simulate adverse real-world conditions such as non-uniform illumination and water vapor. An MV-CA003-21UM Hikvision industrial camera was used to capture vibration videos of the simply supported beam, and the algorithm successfully extracted the global displacement response. The comparison between the identified vibration displacement and the measurements from the eddy current displacement sensor showed an error of approximately 5%, reassuring the algorithm’s accuracy. Moreover, the algorithm accurately identified multiple modal parameters of the simply supported beam, confirming its effectiveness. Finally, this study applied the method to identify vibration displacement and modal parameters of a real reinforced concrete beam, reinforcing its potential application in practical engineering environments.

Keywords

structural health monitoring machine vision vibration measurement image interpolation gray-scale projection

Introduction

Real-time monitoring and evaluation of the health status of bridges are crucial for ensuring their long-term stability. By continuously monitoring the real-time dynamic response of bridges, it becomes feasible to extract the actual dynamic characteristic parameters of the bridges, thereby understanding the structural condition. Acceleration data is commonly used to identify dynamic characteristics in practical engineering applications (Catbas et al., 2008; Wang et al., 2014). This method has developed over the years and provides high-precision identification capabilities. However, as a traditional contact-based response measurement method, acceleration sensors only achieve single-point measurements. Therefore, to obtain comprehensive multi-mode modal information, multiple acceleration sensors must be deployed on the structure, increasing workload and costs and becoming challenging in complex and harsh environments, especially in hard-to-reach areas. With the rapid development of computer technology and digital image processing techniques, machine vision-based displacement measurement technology has provided a non-contact and multi-point measurement solution for structures in recent years. Its application in the field of bridge monitoring is becoming increasingly popular. This method overcomes the limitations of traditional acceleration measurements and provides a more flexible and efficient monitoring approach in complex environments.

The fundamental principle of machine vision-based vibration displacement measurement involves analyzing the image sequences captured before and after structural deformation in videos. The positions of targets in the digital image sequences are matched and tracked through image processing algorithms to obtain vibration information about the structure (Park et al., 2015). Most laboratory research and engineering examples currently use expensive high-resolution industrial cameras to obtain accurate displacement measurement data. Xu et al. (2021) combined a Siamese tracker based on deep learning with correlation-based template matching to propose a new machine vision displacement measurement method, achieving measurements of object reciprocating motion. The experiment utilized a Hikvision industrial camera (resolution 2592 × 2048, maximum frame rate 71.8 fps) to capture vibration images. Lu et al. (2023) used an industrial camera with a resolution of 2592 × 1944 and a frame rate of 24 fps to perform 24-h monitoring of static displacement under varying environmental lighting conditions. This method, which combines deep learning and digital image processing techniques, effectively overcomes the impact of environmental lighting changes on vibration identification. Lee et al. (2020) proposed using a dual-camera system to compensate for camera self-motion errors during monitoring. They conducted indoor experiments using four different resolution cameras: 1920 × 1200, 3840 × 2400, 7680 × 4800, and 15360 × 9600 (all with a frame rate of 163 fps). The results showed that as the camera resolution increased, the errors caused by camera self-motion were significantly suppressed. Cai et al. (2023) applied a non-contact monocular vision system based on the target tracking algorithm (KCF) to monitor displacement in a dual-column vibration table test. The experiment used an industrial camera with a resolution of 2448 × 2048 and a frame rate of 120 fps, demonstrating that the method exhibited high accuracy and robustness in identifying vibration displacement. The aforementioned studies have achieved remarkable effectiveness in measuring machine vision-based displacement.

While high-definition industrial cameras can provide high-quality images, they are often very expensive, thus increasing testing costs. Moreover, high-definition cameras typically have a lower sampling frequency, which may introduce ghosting issues during subsequent image processing, affecting the final measurement accuracy. Furthermore, high resolution can impact system robustness, require significant computational time, and pose challenges for real-time vibration monitoring of structures. Therefore, researchers have started exploring techniques to enhance image resolution that can use lower-cost industrial cameras with relatively lower pixel counts to capture the dynamic displacement of structures. Fukuda et al. (2013) developed the Orientation Code Matching (OCM) technique and reduced measurement errors of a low-resolution camera (resolution 640 × 480 pixels, frame rate 60 fps) using bilinear interpolation. Luo et al. (2018) combined template matching algorithms and sub-pixel interpolation techniques to monitor bridge vibration displacement using a POINT GREY industrial camera (resolution 1280 × 1024 pixels, frame rate 150 fps) to test the dynamic response of the Manhattan Bridge. Ye et al. (2016) proposed a method for synchronous multi-point measurement of structural dynamic displacement. They studied indoor experiments to investigate the effect of measurement distance on the accuracy of displacement identification. The experiment used a Prosilica GigE GE1050 industrial camera (resolution 1024 × 1024 pixels, frame rate 59 fps) as the acquisition device. These studies demonstrate that through image resolution enhancement techniques, high-precision dynamic displacement measurements can be achieved using lower-cost industrial cameras, providing new opportunities for practical engineering applications.

In recent years, with the advancement of related technologies, there has been further research on image processing and displacement recognition. Zhao et al. (2019) combined the Support Correlation Filter (SCF) and Kanade-Lucas-Tomasi (KLT) algorithm to measure the vibration displacement of a cable-stayed bridge model. They validated the accuracy of the method by comparing it with the results from laser displacement sensors. Yang et al. (2019) utilized blind source separation, edge detection, and image magnification techniques to analyze full-field optical flow, effectively extracting the modal parameters of cable vibrations from video data. Additionally, deep learning methods have been applied to displacement recognition with promising results. Dong et al. (2020) proposed a full-field optical flow method based on deep learning, using pre-trained deep neural networks for optical flow computation, which was employed for displacement monitoring in grandstand structures. Zhang et al. (2021) proposed a structural displacement monitoring method based on mask regions with convolutional neural network (Mask R-CNN), This method combines deep learning with computer vision to extract the coordinates of the calibration object.

However, the applicability of these techniques still requires further research. Firstly, traditional template matching techniques have certain requirements for image quality. Matching accuracy is affected under low-resolution image conditions, and template matching methods are sensitive to changes in lighting, partial target occlusion, and background interference. Additionally, optical flow estimation methods are based on the assumption of small displacements and brightness invariance, making it challenging to implement in practical measurements. Secondly, these methods do not fully exploit the lighting and shadow information present in images. Since the objects being measured typically exhibit block-like features with variations in brightness, if these features are effectively utilized after being captured by a low-resolution camera, it can significantly enhance measurement accuracy and stability. Lastly, methods based on deep learning still lack in recognition speed. Further research is needed on how to accurately and rapidly identify and measure structural vibration displacements using low-resolution cameras under limited image resolution conditions.

In response to the aforementioned issues, this study optimized existing machine vision methods and proposed an algorithm that combines ROI image interpolation with gray-scale projection. This method allows for lower-cost, low-resolution, high-frame-rate cameras. The initial step involves the template matching method used to extract the region of interest (ROI) for all measurement points of the target, followed by applying bicubic interpolation within the ROI to enhance image resolution. Subsequently, a gray-scale projection is performed in the direction of vibration to generate unique gray-scale projection features, which can capture dynamic positional changes of measurement points. Moreover, gray-scale projection accumulation enhances light and shadow information in the vibration direction, forming stable and trackable features. This method effectively filters noise and improves accuracy while accelerating computational speed compared to traditional template matching. Hammer impact tests were used on an aluminum alloy beam model bridge in an indoor environment, simulating light changes and fog obstruction that may occur during on-site measurements. Displacement responses were extracted, and modal parameters were identified. Finally, this method was successfully used to measure the vibration of actual reinforced concrete beams, demonstrating its ability to provide accurate and stable measurement outcomes in complex environments and promising application prospects.

Vibration measurement algorithm based on ROI gray-scale projection feature matching

Image preprocessing

For the identification of structural displacement responses, typically only a small portion of the entire image requires focused attention, known as the Region of Interest (ROI). Processing only the ROI can significantly enhance recognition efficiency and accuracy. This study utilizes template matching techniques to locate and extract the ROI, automatically generating vibration measurement areas based on sensor positions. Subsequently, grayscale conversion is applied to the images to improve the speed of subsequent image processing, and interpolation is performed on the ROI to enhance the quality of the original images. Common interpolation methods include nearest-neighbor, bilinear, bicubic, and Lanczos interpolation (Acharya and Meher, 2012). In this study, the bicubic interpolation method is chosen for interpolating the target ROI to avoid affecting computational efficiency by interpolating the entire image. This method helps in reproducing image details, thereby improving the accuracy and efficiency of image processing.

Visual tracking algorithm based on grayscale projection feature matching

Grayscale projection is a commonly used technique in image processing for rapid recognition and tracking of moving objects. It accumulates the grayscale values of the image in a specific direction (usually horizontal or vertical) to extract image features (Baskan et al., 2002). In identifying small displacements, the template image of the measurement point does not have clear boundaries but rather transitions in the form of brightness gradients. This study used grayscale projection to enhance this characteristic, accumulating the grayscale values of each row of pixels in the image along the direction of vibration, forming a one-dimensional grayscale projection feature. The calculation is shown in equations (1) and (2). This method amplifies the grayscale changes of the measurement points, making the features more prominent, as depicted in Figure 1.

\begin{array}{l} P_{r} (i) = \sum_{j = 1}^{W_{r}} G_{r} (i, j) & , & i \in H_{r} \end{array}

(1)

\begin{array}{l} P_{m} (i) = \sum_{j = 1}^{W_{m}} G_{m} (i, j) & , & i \in H_{m} \end{array}

(2)

where P(i) denotes the sum of grayscale values in the i-th row; G(i, j) denotes the grayscale value of pixel (i, j); W is the image width; H is the image height; the subscripts r and m respectively refer to the measurement point area and the measurement area.

Figure 1.

Grayscale projection feature.

P_r denotes the grayscale projection feature of the reference measurement point, which remains unchanged throughout the testing process. P_m denotes the grayscale projection of the vibration measurement ROI. Subsequently, it is only necessary to perform line-by-line matching operations between P_r and P_m and determine the row in P_m with the highest degree of matching with P_r, obtaining the relative positional information of the measurement point in the ROI area. The matching formulas are shown in equations (3) and (4).

R_{m} (k) = \sum_{i = 1}^{H_{r}} {[P_{r} (i) - P_{m} (i + k)]}^{2}, k \in H_{m}

(3)

V_{t} = \arg \underset{k}{m} in [R_{m, t} (k)]

(4)

where k denotes the number of pixel rows from the matching origin; R(k) denotes the degree of matching between P_r and P_m at a distance of k rows from the matching origin in the ROI area; V_t denotes the relative position of the measurement point in the t-th frame.

By calculating frame by frame, the motion vectors V₁, …, V_k, …, and V_n of each measurement point can be obtained, as shown in Figure 2. At time t₀, the system is in a static state, V₀ denotes the position of the measurement point at the static moment, and t_n denotes a frame of the simply supported beam vibration. Figure 2(b) shows the captured images of other objects near the measurement point when extracting the target ROI. However, the features formed by grayscale projection in this area differ significantly from the measurement point features and do not affect the identification results of the measurement points. By performing horizontal grayscale projection on the image, the grayscale projection curve in that direction is obtained, and the relative positions of the peaks of the two curves are calculated to eventually determine the position information of the target measurement point at that moment. The grayscale projection algorithm maximizes the utilization of the light and shadow information contained in the video images, resulting in a more stable matching feature through projection. This reduces computational complexity while ensuring matching accuracy compared to traditional matching algorithms.

Figure 2.

Vibration measurement based on gray-scale projection: (a) no shade within the ROI, (b) Masking within the ROI.

The traditional template matching methods are sensitive to changes in lighting conditions, which often unavoidably affect measurements in practical engineering applications. The method proposed in this paper determines targets through matching grayscale distributions, focusing more on the relative strength of grayscale values at measurement points. As long as the relative strengths of grayscale values remain unchanged, the final results will not be altered. Typically, our measurement range is a relatively small area where lighting variations manifest as overall enhancement or weakening within this range, as illustrated in the Figure 3. Therefore, variations in lighting during the testing process do not affect the relative strengths of grayscale values, thereby not influencing the measurement of the final position of the measurement points.

Figure 3.

Comparison of grayscale projections of the same measurement point under different lighting intensities.

The impact of camera self-vibration

In practical measurements, ground vibrations, wind, and human disturbances often cause slight movements in the camera itself, resulting in errors in the measurement of the target object. This study combines the advantages of multipoint measurements proposed in the method to synchronously measure the target object and fixed objects in the background. The measured displacement of fixed objects in the background is used to account for errors induced by the camera’s self-vibration:

D_{c} = D_{b}

(5)

where D_c denotes the displacement caused by the camera’s self-vibration, and D_b represents the measured displacement of fixed objects in the background. Before converting the measured pixel displacement of the target into actual displacement, it is necessary to subtract the error induced by the camera’s self-vibration (Luo et al., 2018).

D_{r} = D_{t} - D_{c}

(6)

where D_r denotes the actual displacement of the target being measured, and D_t represents the displacement of the target identified in the image.

Camera calibration and scale factor calculation

Once pixel displacement information of structures in video images is obtained, it is essential to establish the relationship between pixel coordinates and real-world coordinates and determine the scale factor between the two to obtain the actual structural displacement. In practical filming scenarios, lenses do not provide ideal perspective imaging, and lens distortion can lead to image distortion. This distortion can be effectively eliminated using the calibration method proposed by (Zhang, 2000).

The scale factor can be obtained in two ways (Cai et al., 2023; Feng and Feng, 2018; Lei et al., 2021; Lu et al., 2023; Zhu et al., 2021), as illustrated in Figure 4: (1) When the camera optical axis is perpendicular to the plane of object motion, based on the known physical dimensions on the object surface and their corresponding image dimensions, the scale factor SF₁ can be expressed as:

S F_{1} = \frac{D}{d}

(7)

where D denotes the actual size of the object, and d is the pixel size of the object in the image.

Figure 4.

Calculation diagram of scale factor.

(2) Based on the intrinsic parameters of the camera and the external parameters between the camera and the object structure, the scale factor SF₂ can be expressed as:

{S F}_{2} = \frac{Z}{f} d

(8)

where Z denotes the distance between the camera and the object, and f is the focal length. When there is a tilt angle

θ

between the camera optical axis and the normal to the object surface, the scale factor SF₃ can be expressed as:

{S F}_{3} = \frac{Z}{f c o s^{2} θ} d

(9)

where

θ

denotes the tilt angle between the camera optical axis and the normal to the object surface. Since this study utilizes industrial cameras and lenses as image acquisition devices, we can disregard image distortion occurring during the imaging process. Additionally, the camera optical axis is perpendicular to the plane of motion of the object being measured. Therefore, in this scenario, method (1) is used to determine the scale factor.

Measurement method and steps

This research is focused on using the Python language and the computer vision library OpenCV to develop an algorithm for measuring machine vision vibration and recognizing modal parameters based on grayscale projection. It processes structural vibration images to extract their displacement responses and identify modal parameters. Figure 5 illustrates the measurement process, and the specific steps are as follows:

(1) Use template matching to extract the desired Region of Interest (ROI) and ascertain the range of all measurement points for the vibration test.

(2) Apply bicubic interpolation and grayscale projection on the vibration image ROI to create grayscale projection features. Match the grayscale projection features of the measurement points with those of the ROI to track dynamic changes in the positions of the measurement points.

(3) Convert the pixel displacements of the structure in the image to real displacements of the structure using a scaling factor.

(4) Extract vibration displacement data obtained through machine vision methods and eddy current sensors, plot displacement time history curves, and compare them.

(5) Use the collected structural time-domain displacement signals to identify modal parameters and then compare them with the results obtained from numerical simulation.

Figure 5.

Flowchart of the measurement and modal identification process.

Simply supported aluminum beam hammer test

To evaluate the accuracy and precision of the machine vision measurement method proposed in this study, a simply supported aluminum alloy beam model was established in the laboratory, and the vibration of the beam was excited using an impact method. Figure 6 presents the experimental setup, with monitoring points and visual targets on the model to assist in machine vision vibration recognition. Furthermore, three detection points labeled 5, 10, and 19 were equipped with eddy current displacement sensors to acquire vibration displacement data, which was then compared with the vibration recognition data obtained through the machine vision method.

Figure 6.

Experimental equipment and models: (a) overall layout diagram, (b) object-image distance, and (c) camera arrangement.

During the collection of vibration video images, a tripod stabilized the camera to avoid measurement errors caused by the camera shake. Moreover, to ensure the industrial camera captured clear images and obtained accurate vibration data, indoor lighting was turned on, and camera exposure increased. As depicted in Figure 6, the industrial camera was positioned 2.1 m away from the simply supported beam. This experiment did not consider the torsion of the simply supported beam, focusing solely on the vibration of the beam in the y-z plane.

Experimental cases

A simply supported aluminum beam model was constructed in the laboratory, with horizontal and vertical constraints at the left and right ends. The beam had a length of 1100 mm and a thickness of 5 mm. Table 1 displays the material properties and geometric parameters of this specific model. Figure 7 depicts the arrangement of 20 machine vision detection points on the simply supported beam. Furthermore, eddy current displacement sensors were arranged at three key positions on the simply supported beam: detection point 5 at 1/4 distance from the left support, detection point 10 at the midspan, and detection point 19 at the right edge. Detection point 19, located near the support, has a smaller vibration amplitude, requiring higher measurement accuracy and presenting greater measurement challenges. Vibration measurements at this position are crucial for validating the accuracy and robustness of the proposed algorithm. By conducting measurements under these complex conditions, we can comprehensively evaluate the performance of the algorithm by handling low-amplitude vibrations and high-demand measurement points, demonstrating its reliability and effectiveness in practical applications.

Table 1.

The Material Properties and Geometric Parameters of a Simply Supported Beam Model.

Elastic modulus (Pa)	6.85 × 10¹⁰
Poisson’s ratio	0.33
Length (m)	1.1
Width (m)	0.05
Heigh (m)	0.00505
Moment of inertia (m⁴)	5.4 × 10⁻¹⁰
Density (kg/m³)	2.7 × 10³

Figure 7.

Measurement point layout diagram.

Two scenarios—uneven lighting and steam obstruction—were considered to investigate the testing accuracy of the machine vision method proposed in this study for real engineering environments. These scenarios were simulated during the experiments using supplementary lighting and a humidifier, with a normal testing condition without interference selected as the control, as depicted in Figure 8. Three test cases were established, as listed in Table 2. During testing, the frame rate for data acquisition of the industrial camera and eddy current sensors was set to 500 fps.

Figure 8.

The layout diagrams for the three cases. (a)Normal, (b) Uneven illumination, (c) Water vapor obstruction.

Table 2.

Experimental Condition Design.

Case	Interference factors	Number of measurement points
1	Normal	20
2	Uneven illumination	20
3	Water vapor obstruction	20

Composition of measurement system

The method proposed in this study uses the MV-CA003-21UM Hikvision industrial camera for capturing images, as shown in Figure 9. The sensor is a CMOS type with a global shutter sensor model PYTHON300; it has a resolution of 640 × 480 and can achieve a maximum sampling frame rate of 814 fps, and a dynamic range of 59 dB. The camera lens used is the M1806-5MP industrial lens with a fixed focal length of 6 mm, as illustrated in Figure 10. Furthermore, eddy current displacement sensors were set up for comparative validation, with the sensor model being TST3826F-HW dynamic-static strain testing analysis system, as depicted in Figure 11.

Figure 9.

Industrial camera module.

Figure 10.

Camera lens module.

Figure 11.

Displacement sensor module.

Analysis of experimental results

Vibration displacement identification results

We analyzed the vibration measurement results of the measurement points under each working condition according to the above test method. To validate the accuracy of machine vision detection, displacement-time curves were plotted by detecting the vibration displacement of the measurement points and compared with the displacement-time curves obtained by the eddy current sensor. Due to the large number of measurement points, this study specifically selected the measurement points with the maximum and minimum vibration amplitudes in each of the three working conditions. Specifically, measurement points 10 and 19 were chosen for presentation.

Figure 12 demonstrates that the displacement-time graph of the measurement points in pixel coordinates was obtained by tracking their dynamic position change information. This was achieved by matching the features of the grayscale projection within the ROI (the blue area on the right side of the image). The displacement in pixel coordinates was normalized and compared with the data results from the eddy current displacement sensor. For illustrative purposes, we used measurement point 19 from working case 1 as an example, as presented in Figure 13.

Figure 12.

Schematic of machine vision vibration recognition.

Figure 13.

Normalized displacement time plot at measurement point 19 for Case 1.

Figure 13 illustrates that the vibration displacements obtained by the two methods are generally similar, but significant errors are present at peak values, with the sensor-generated displacement curve appearing smoother. On one hand, the actual measurement range of the eddy current sensor exceeds the width of the detection points, as depicted in Figure 14, leading to a spatial averaging effect in the sensor’s results. On the other hand, the machine vision method introduces some noise. In this study, a sliding average filter was applied to individual machine vision measurement points to mitigate noise (Figure 15):

S M A_{t} = \frac{P_{1} + P_{2} + \dots + P_{n}}{n}

(10)

where SMA_t denotes the simple moving average result; P denotes the displacement data; n refers to the number of time sequences used for the moving average.

Figure 14.

Measuring range of eddy current sensors and range of measurement points.

Figure 15.

Comparison of machine vision and sensor results after simple sliding average.

To determine the final actual structural displacement, it is crucial to establish the scale factor (SF) between pixel coordinates and actual coordinates once the displacement information of the measurement points has been obtained. Since this study utilized an industrial camera as the image acquisition device and could ignore image distortion during the imaging process, a simplified method was adopted to calculate the scale factor. This method is based on the known physical dimensions of the object surface and their corresponding image dimensions.

In this study, the scale factor was calculated by comparing the specific dimensions of a known calibration target with the image pixel dimensions. A chessboard calibration target with a side length of 15 mm was used as the target. After fixing the position, the tilt angle and focal length of the industrial camera were fixed, and the calibration board was placed at the target measurement point and photographed. A chessboard region near the target measurement point was selected to reduce errors. The cv2.goodFeaturesToTrack function was used to identify the corner points of the chessboard, and the average distance between the corner points in the pixel coordinate system was calculated. The scale factor corresponding to that point was then determined using equation (7), as depicted in Figure 16.

Figure 16.

Checkerboard calibration.

Subsequently, the structural displacements obtained in pixel coordinates through machine vision recognition were converted into real displacements and then compared with the displacement-time curves measured by the eddy current sensor for further analysis. Figures 17 and 18 present the experimental results.

Figure 17.

Comparison of machine vision and eddy current sensors for measurement point 10. (a) Case 1, (b) Case 2, (c) Case 3.

Figure 18.

Comparison of machine vision and eddy current sensors for measurement point 19. (a) Case 1, (b) Case 2, (c) Case 3.

The results showed that the displacement-time curve measured by the machine vision method proposed in this study closely matches the displacement-time curve obtained by the displacement sensor. However, some discrepancies were observed at the peak values. This study used the average frame-wise squared error (AFSE) and the normalized root mean square error (NRMSE) for error analysis. The mathematical expressions are as follows:

AFSE = \frac{1}{n} \sum_{i = 1}^{n} \frac{| x_{i} - y_{i} |}{y_{\max} - y_{\min}} \times 100 %

(11)

NRMSE = \frac{\sqrt{\frac{1}{n} \sum_{i = 1}^{n} {(x_{i} - y_{i})}^{2}}}{y_{\max} - y_{\min}} \times 100 %

(12)

where n is the number of data points; x_i (mm) denotes the i-th data value of machine vision vibration displacement measurement obtained at time t; y_i (mm) denotes the i-th data value of displacement measured by the displacement sensor at time t; y_max (mm) is the maximum displacement value measured by the displacement sensor; y_min (mm) is the minimum displacement value measured by the displacement sensor.

To validate the effectiveness of the proposed method under low-resolution conditions and interference from lighting and fog, a comparison was made between the method presented in this paper and a subpixel-enhanced template matching algorithm (Feng et al., 2015). The results are shown in Table 3, displays the AFSE and NRMSE values for each point in Cases 1–3. The total AFSE for the nine detection points for the three cases is approximately 3.29%, while the NRMSE is around 5.12%. These results indicate a good agreement between the machine vision measurement results and the data from the eddy current sensor, confirming the accuracy of the proposed machine vision vibration displacement measurement method that meets the precision requirements for vibration displacement measurement. Moreover, compared to the subpixel-enhanced template matching method, the grayscale projection algorithm has demonstrated higher accuracy and robustness. It can perform precise displacement measurements even under lower video image resolution conditions.

Table 3.

Tolerance for Different Cases.

Case	Measurement point	AFSE (%)		NRMSE (%)
Case	Measurement point	Our	Subpixel-enhanced	Our	Subpixel-enhanced
1	5	3.00	6.81	4.72	9.79
	10	3.49	5.94	5.69	9.85
	19	3.33	6.15	4.96	9.57
2	5	3.50	6.24	5.34	9.54
	10	2.63	8.13	4.47	14.06
	19	4.11	7.69	5.90	10.90
3	5	3.54	7.11	5.03	10.37
	10	2.32	6.15	4.66	9.28
	19	3.69	7.21	5.35	10.21

Modal parameter identification results

This study used the covariance-based subspace system identification (COV-SSI) method (Kvåle et al., 2017; Sun et al., 2023) to obtain the modal parameters of the structure, such as frequency and mode shapes, using the vibration displacement identified by machine vision. Figure 19 presents the power spectral density plots of measurement point 10 compared to each measurement point. For example, the modal mode shapes identified in Case 1 are shown in Figure 20.

Figure 19.

CPSD of point 10 and other points for three cases. (a) Case 1, (b) Case 2, (c) Case 3.

Figure 20.

Shapes of three modal. (a) Mode 1, (b) Mode 2, (c) Mode 3.

Subsequently, a finite element model of a simply supported aluminum beam was established using OpenSeesPy, with its geometric parameters and material properties shown in Table 1. The frequencies of numerical simulation of the simply supported beam were identified and compared with the results obtained from the machine vision method and displacement sensor identification, as shown in Tables 4 –6. Analyzing the data in Tables 4 –6, it was found that in Case 1, the maximum difference between the frequencies identified by the machine vision method and the finite element simulation results is 3.42%, whereas the minimum is 1.92%; in Case 2, the maximum error is 3.45%, whereas the minimum is 1.35%; in Case 3, the maximum error is 3.45%, whereas the minimum is 1.63%. Furthermore, the modal assurance criterion (MAC) values for the first three mode shapes identified by the machine vision method and the finite element results are all greater than 0.95. This result indicates that the accuracy of the modal parameter identification for the machine vision method proposed in this study meets the experimental requirements.

Table 4.

Frequency Comparison of Finite Element Data With Machine Vision and Displacement Sensor Data for Case 1.

Mode	Frequency from FEM (Hz)	Machine vision		Displacement sensor
Mode	Frequency from FEM (Hz)	Frequency (Hz)	Difference (%)	Frequency (Hz)	Difference (%)
1	10.153	10.348	1.92	10.442	2.85
2	40.604	39.216	3.42	39.168	3.53
3	91.341	88.344	3.28	83.332	8.77

Table 5.

Frequency Comparison of Finite Element Data With Machine Vision and Displacement Sensor Data for Case 2.

Mode	Frequency from FEM (Hz)	Machine vision		Displacement sensor
Mode	Frequency from FEM (Hz)	Frequency (Hz)	Difference (%)	Frequency (Hz)	Difference (%)
1	10.153	10.314	1.59	10.420	2.63
2	40.604	39.252	1.35	39.202	3.45
3	91.341	88.186	3.45	83.342	8.76

Table 6.

Frequency Comparison of Finite Element Data With Machine Vision and Displacement Sensor Data for Case 3.

Mode	Frequency from FEM (Hz)	Machine vision		Displacement sensor
Mode	Frequency from FEM (Hz)	Frequency (Hz)	Difference (%)	Frequency (Hz)	Difference (%)
1	10.153	10.318	1.63	10.408	2.51
2	40.604	39.232	3.38	39.180	3.51
3	91.341	88.218	3.42	83.350	8.75

In practical engineering measurements, the structures under test are typically long. In such cases, a method involving multiple measurements at local points can be employed. Ultimately, vibration measurement and mode identification for the entire structure can be achieved through modal stitching of overlapping sections.

Vibration measurement of a physical simply supported beam

To further validate the effectiveness of this method in practical engineering environments, this chapter conducted vibration measurements on a reinforced concrete (RC) simply supported beam under natural lighting conditions. Modal parameters for the structure were identified using the machine vision method and the displacement sensor signal, and then they were compared. Subsequently, finite element results were provided as a reference.

Description of the specimen

The test specimen is a reinforced concrete beam with overall dimensions of 9000 × 800 × 150 mm³. The concrete has a grade of C40 and is reinforced longitudinally with 10 HRB400 steel bars measuring a diameter of 16 mm. Embedded fixtures are installed at the beam ends, and the top surface is covered with PVC rubber. Figure 21 depicts the overall view of the RC beam. The beam has experienced significant creep deformation, exhibiting substantial deformation before testing. The specimen dimensions and reinforcement details are illustrated in Figure 22.

Figure 21.

Realistic view of the RC beam.

Figure 22.

External geometric dimensions and reinforcement arrangement of test beam (unit: mm): (a) site plan; (b) elevation plan.

Experimental cases and layout of measurement points

To determine the modal parameters of the simply supported RC beam, the beam was placed on elastomeric bearings with both ends simply supported. Hammer excitation was applied to the surface of the beam at a distance of 1.5 m from the left support to obtain the vertical displacement response under natural lighting conditions. A visual target was positioned at regular intervals of 1/8th of the length of the beam as a measurement target for machine vision, with seven machine vision measurement points. An industrial camera was positioned with its axis perpendicular to the vibration plane of the beam at a distance of 5.2 m from the mid-span of the beam. Due to limitations in equipment focal length and resolution, the camera could not capture all measurement points simultaneously. Therefore, the measurement was conducted in three intervals: points 1, 2, and 3 were measured in the first interval, points 3, 4, and 5 in the second interval, and points 5, 6, and 7 in the third interval. Furthermore, displacement sensors were positioned at distances 1/8th and 1/4th from the beam end, as well as at the mid-span (i.e., measurement points 1, 2, 4, 6, 7) to verify and compare the accuracy of machine vision vibration displacement measurements. To achieve this objective, eddy current sensors were used, and thin iron plates were attached to the bottom surface of the beam at the corresponding measurement points to serve as the displacement measurement targets for the eddy current sensors. Figure 23 depicts the on-site instrument layout, and a detailed layout of the measurement points is depicted in Figure 24.

Figure 23.

Arrangement of instruments.

Figure 24.

Layout of measurement points.

Experimental results

Displacement identification results

The vibration displacement measured by machine vision at detection points 1, 2, 4, 6, and 7 was compared with the vibration displacement measured by displacement sensors. Figure 25 compares the vibration displacement at different detection points. Table 7 depicts the errors between the machine vision vibration displacement measurement and the displacement sensor measurement at the five detection points. The average AFSE for the five detection points is approximately 5.15%, and the average NRMSE is approximately 6.69%. Compared to indoor experiments (with an average AFSE of approximately 3.29% and an average NRMSE of approximately 5.12%), the errors in vibration displacement measurement are greater but remain within the acceptable range.

Figure 25.

Comparison of displacement between machine vision and displacement sensors. (a) Point 1, (b) Point 2, Point 4, (d) Point 6, (e) Point 7.

Table 7.

Error at Different Detection Points.

Measurement point	AFSE (%)	NRMSE (%)
1	3.64	5.03
2	3.89	5.68
4	7.74	9.51
6	3.71	5.35
7	6.77	7.86

Modal parameter identification results

Similar to section 3.3.2, the modal parameters of the beam structure were determined using the COV-SSI method based on the vibration displacement obtained by the machine vision approach. The power spectral density was computed for each measurement point, starting with point 4, as depicted in Figure 26. Subsequently, the modal parameters of the simply supported beam were combined to determine the first three mode shapes of the structure, as illustrated in Figure 27.

Figure 26.

CPSD of point 3 and other points.

Figure 27.

Shapes of three modal. (a) Mode 1, (b) Mode 2, (c) Mode 3.

Subsequently, a finite element analysis model of the RC beam was established using the software Abaqus CAE (2020), as shown in Figure 28. The entire beam was divided into 43,551 elements and 49,869 nodes, with the reinforcement simulated using truss elements. A general contact approach was used to simulate the connection between the beam and the plate rubber bearing, with a friction coefficient of 0.3. The Lanczos method was employed in the finite element model to calculate the frequencies of the RC beam at each mode.

Figure 28.

Finite Element Analysis Model of RC beam.

The vibration frequencies of the simply supported beam were determined using the machine vision method and compared with the results obtained from the eddy current sensor. Table 8 provides the finite element simulation results, indicating that the errors of the first three modal frequencies obtained by the machine vision method in this study compared to the displacement sensor identification results are all about 5%, with an average error of 4.49%, meeting the accuracy. The results indicate that the method proposed in this study can be applied in practical engineering environments.

Table 8.

Frequency Comparison Between Finite Element Data and Machine Vision Data.

Mode	Machine vision	Displacement sensor		FEM (Hz)
Mode	Machine vision	Frequency (Hz)	Difference (%)	Frequency (Hz)
1	2.389	2.314	3.24	2.402
2	9.966	9.468	5.53	9.876
3	22.082	21.091	4.70	22.388

Conclusions

In this study, we used a low-resolution, high-frame-rate industrial camera to identify structural vibration displacement and modal parameters. The method is validated through laboratory-supported beam impact tests and on-site tests of RC beams, leading to the following conclusions:

(1) The proposed method used grayscale projection to form stable matching features, fully exploiting the light and shadow information in the images. It also achieved high accuracy in measuring structural vibration displacement and identified modal parameters under low-resolution conditions while improving computational speed.

(2) In the aluminum beam impact test, the displacement time history curves obtained by the machine vision method and the eddy current sensor exhibited good agreement, with a certain level of error at the peak, resulting in AFSE and NRMSE values of around 5%. Subsequently, applying a fast Fourier transform to the displacement data and using the COV-SSI method to obtain structural vibration frequencies demonstrated good agreement between the proposed method and the finite element numerical simulation. The MAC values are all above 0.95, indicating the ability to effectively identify multiple modal orders of the target.

(3) Vibration measurement and modal parameter identification of RC beams were conducted in actual environmental conditions using the machine vision method. The proposed method demonstrated good alignment in identifying actual vibration displacement, with errors in the detected modal frequencies and displacement sensor values around 5%, slightly higher than those in indoor tests. Nevertheless, the method still meets the accuracy criteria for measuring vibration displacement measurement, confirming its feasibility in real-world engineering conditions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is jointly funded by the: National Natural Science Foundation of China Youth Science Fund (No. 52308531), New Interdisciplinary Cultivation Project of Southwest Jiaotong University (No. 2682022KJ054), Research project of Sichuan Jiaoda Engineering Testing Consulting Co., Ltd (R110120H01506), Natural Science Foundation of Sichuan Province (2025ZNSFSC0410).

ORCID iD

Shengyu Li

References

Acharya

Meher

(2012) Region adaptive unsharp masking based Lanczos-3 interpolation for video intra frame up-sampling. In: 2012 Sixth International Conference on Sensing Technology (ICST). IEEE, 57–62.

Baskan

Bulut

Atalay

(2002) Projection based method for segmentation of human face and its evaluation. Pattern Recognition Letters 23(14): 1623–1629.

Cai

Xie

(2023) KCF-based identification approach for vibration displacement of double-column bents under various earthquakes. Structural Control and Health Monitoring 2023(1): 1–22.

Catbas

Susoy

Frangopol

(2008) Structural health monitoring and reliability estimation: long span truss bridge application with environmental monitoring data. Engineering Structures 30(9): 2347–2359.

Dong

C-Z

Celik

Catbas

, et al. (2020) Structural displacement monitoring using deep learning-based full field optical flow methods. Structure and Infrastructure Engineering 16(1): 51–71.

Feng

(2018) Computer vision for SHM of civil infrastructure: from dynamic response measurement to damage detection–A review. Engineering Structures 156: 105–117.

Feng

Ozer

, et al. (2015) A vision-based sensor for noncontact structural displacement measurement. Sensors 15(7): 16557–16575.

Fukuda

Feng

Narita

, et al. (2013) Vision-based displacement sensor for monitoring dynamic response using robust object search algorithm. IEEE Sensors Journal 13(12): 4725–4732.

Kvåle

Øiseth

Rønnquist

(2017) Operational modal analysis of an end-supported pontoon bridge. Engineering Structures 148: 410–423.

10.

Lee

K-C

Jeong

, et al. (2020) Long-term displacement measurement of full-scale bridges using camera ego-motion compensation. Mechanical Systems and Signal Processing 140: 106651.

11.

Lei

Song

Liu

, et al. (2021) Displacement identification by computer vision for condition monitoring of rail vehicle bearings. Sensors 21(6): 2100.

12.

Bai

Zhao

(2023) Vision-based structural displacement measurement under ambient-light changes via deep learning and digital image processing. Measurement 208: 112480.

13.

Luo

Feng

(2018) Robust vision sensor for multi-point displacement monitoring of bridges in the field. Engineering Structures 163: 255–266.

14.

Park

Kim

, et al. (2015) 3D displacement measurement model for health monitoring of structures using a motion capture system. Measurement 59: 352–362.

15.

Sun

Yang

Zhang

, et al. (2023) Fast online implementation of covariance-driven stochastic subspace identification. Mechanical Systems and Signal Processing 197: 110326.

16.

Wang

Tao

Guo

, et al. (2014) Full-scale measurements and system identification on sutong cable-stayed bridge during typhoon fung-wong. The Scientific World Journal 2014(1): 936832.

17.

Zhang

Brownjohn

(2021) An accurate and distraction-free vision-based structural displacement measurement method integrating Siamese network based tracker and correlation-based template matching. Measurement 179: 109506.

18.

Yang

Sanchez

Zhang

, et al. (2019) Estimation of full-field, full-order experimental modal model of cable vibration from digital video measurements with physics-guided unsupervised machine learning and computer vision. Structural Control and Health Monitoring 26(6): e2358.

19.

T-H

Dong

, et al. (2016) Vision-based structural displacement measurement: system performance evaluation and influence factor analysis. Measurement 88: 372–384.

20.

Zhang

(2000) A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(11): 1330–1334.

21.

Zhang

Liu

Zhao

(2021) Structural displacement monitoring based on mask regions with convolutional neural network. Construction and Building Materials 267: 120923.

22.

Zhao

Bao

Guan

, et al. (2019) Video-based multiscale identification approach for tower vibration of a cable-stayed bridge model under earthquake ground motions. Structural Control and Health Monitoring 26(3): e2314.

23.

Zhu

Zhang

, et al. (2021) A multi-resolution deep feature framework for dynamic displacement measurement of bridges using vision-based tracking system. Measurement 183: 109847.