Performance of videogrammetric displacement monitoring technique under varying ambient temperature

Abstract

There has been an increasing number of attempts to apply videogrammetric technique to displacement measurement of civil engineering structures. Its potentials in structural health monitoring have also gained more attention. This study carried out an investigation on the effect of temperature variation on the measurement accuracy of videogrammetric technique in an effort to examine its feasibility for structural health monitoring. Long-term indoor videogrammetric measurement tests have been conducted, and the performance of the videogrammetric displacement monitoring technique under ambient temperature conditions has been examined. The results show that temperature variations cause non-negligible errors in measured displacements. In line with the temperature variation, the displacement measurement error also contains not only daily fluctuation pattern but also overall trend. In terms of daily fluctuation pattern, the horizontal measurement error and temperatures of vision measurement system are in satisfactory consistency, while the vertical measurement error does not coincide well with temperatures of vision measurement system. In terms of overall trend, the vertical measurement error is highly correlated with temperatures of vision measurement system, while the horizontal one is almost uncorrelated with temperatures of vision measurement system. As an outcome of the dominance of overall trend in the temperature variation over a long time period, the vertical measurement error and temperatures of vision measurement system conform to a favorable linear relationship, while the horizontal measurement error tends to be constrained in a small range when the temperatures of vision measurement system exceed a certain value.

Keywords

displacement environmental effect indoor environment structural health monitoring temperature vision measurement system

Introduction

Over the last few decades, videogrammetry has seen many advances benefiting from the exceptional advances in image sensors, computers, and image processing algorithms. Its applications can be found in many diverse fields such as experimental mechanics (Sharpe, 2008) and aerospace engineering (Liu et al., 2012), among others. Thanks to the non-contact nature, it has also allowed new opportunities for monitoring the displacement of a civil engineering structure. In general, steps to realize videogrammetric displacement monitoring include camera calibration, object tracking, and coordinate reconstruction. The camera calibration involves the estimation of intrinsic and extrinsic parameters of a camera. The former provides the relationship between the image and the camera, and the latter provides the relationship between the camera and the real world. For object tracking, templates of the user-selected object features are extracted from the initial frame and a template matching operation is applied repeatedly with the help of an object tracking algorithm. In the three-dimensional measurement, the space coordinates of an object can be identified by making use of the collinearity and space intersection theory, whereas the homography mapping between image coordinates and space coordinates can be used in the two-dimensional case. The displacements of an object are finally determined by tracking the object in different time stages and comparing its coordinates to the reference stage. In the past decades, videogrammetric displacement monitoring has become the subject of intensive research in the civil engineering community. A diversity of videogrammetric displacement monitoring techniques has been proposed and the applications in real civil engineering structures have also been reported (Busca et al., 2014; Chang and Xiao, 2010; Feng and Feng, 2016, 2017; Lee et al., 2007; Lu et al., 2017; Olaszek, 1999; Park et al., 2010, 2015; Wahbeh et al., 2003). To gain a comprehensive view of the state of the art of the videogrammetric displacement measurement technique, refer to several review papers (Baqersad et al., 2017; Jiang et al., 2008; Wu and Casciati, 2014). As evidenced by previous studies, videogrammetric technique shows great potential in the field of civil engineering and it is being applied to an ever-increasing range of tasks.

Nevertheless, there is still not a turn-key system available to civil engineers. Generally, in the harsh environment of field monitoring, the performance of the vision measurement system might strongly deviate from the behavior defined during laboratory testing, which necessitates investigations into the environmental effect on videogrammetric technique. It becomes more important as the potentials of the technique in structural health monitoring have gained more attention (Feng and Feng, 2016, 2017; Lee et al., 2007; Park et al., 2015). In this case, the surrounding environment of the vision measurement system will fluctuate significantly over a long time period, which may induce intolerable errors in the measurement results. So far, a few studies have been reported in this regard and most of these studies focus on the temperature effect. On one hand, the dark current, which is a noise source intrinsic to the image sensor, is strongly temperature dependent (Widenhorn et al., 2002). On the other hand, temperature variations induce thermal deformation of the vision measurement system, resulting in changes in the intrinsic and extrinsic parameters of a camera and subsequently the virtual drifts of image. Robson et al. (1993) carried out an investigation into the suitability of charge-coupled device (CCD) cameras for videogrammetric measurement. Image drift was observed during camera warm-up, which was referred to as warm-up effect. Handel (2009) focused on the study of camera warm-up effect on image acquisition. He stated that the image drift was originated from a slight displacement of the image sensor due to the thermal expansion of the mechanical components of a camera. In addition to this, Yu et al. (2014) argued that temperature variations also induced changes in the refractive index of the lens optical material, which led to changes in the focal distance and optical axis and thereby the drifts of image. Zhou et al. (2017) investigated the performance of the vision measurement system in air-conditioned environments. The displacement measurement error showed an obvious periodic variation pattern, which was similar to that of indoor air temperature, evidencing that temperature variations did cause significant measurement errors. Assuming that temperature variation induces the translation of a camera only while the intrinsic parameters remain unchanged, Podbreznik and Potočnik (2012) proposed a modified analytical camera model to eliminate the temperature-induced error in the videogrammetric measurement. Nevertheless, the actual mechanism responsible for the temperature effect may not be as simple as this. Similarly, Yu et al. (2014) established an image drift model that correlated the drift of coordinates with the variation in camera parameters, and then formulated the relationship between camera parameters and temperature variation. The variation of principal point, which was one of the intrinsic parameters of a camera, was identified as the major cause of image drift. Nevertheless, it could be less well modeled, which decreased the usefulness of the model. As it is seen, prior studies on the temperature effect are still very limited. Particularly, the videogrammetric measurement tests were conducted in air-conditioned environments and lasted a short period of time only in these previous studies. The performance of the vision measurement system under varying ambient temperature, as is often the case in field monitoring, remains unexplored.

This study resorted to a data-driven approach to investigate the temperature effect on the videogrammetric technique under ambient temperature conditions, focusing mainly on the feature extraction of the temperature-induced measurement error. First, long-term videogrammetric measurement tests, which lasted intermittently for more than half a year, were carried out in indoor environments so as to shield off other environmental factors. Temperatures of vision measurement system were monitored meanwhile. Making use of the long-term measurement data, the characteristics of the displacement measurement errors were then examined. Correlation analyses between the displacement measurement error and temperatures of vision measurement system were performed to quantify the degree of correlation. After that, wavelet analysis was employed to decompose them into periodic component and trend component, and the characteristics of the decomposed components were also examined. Finally, making use of the identified features of the temperature-induced measurement error, the possible means to eliminate the error was explored in the context of the deformation monitoring of a civil engineering structure.

Experimental setup

Figure 1 shows the experimental setup. The videogrammetric measurement tests have been conducted inside a single-story laboratory without a basement. It can provide a single undivided indoor space of 81.2 m length, 25.3 m width, and 7.9 m height. The indoor space was kept closed and seldom interfered by human activities over the whole testing period so as to shield off other environmental factors as much as possible. The vision measurement system comprises a digital camera in conjunction with a zoom lens for image acquisition and a laptop for image processing. The digital camera is a monochrome 1/2″ CCD with GigE vision interface made by Allied Vision Technologies, Germany. It has a resolution of 1024 × 1024 pixels and a maximum frame rate up to 60 fps. The lens is a 12× zoom lens manufactured by Navitar Industries, USA. Its focal length can be adapted in accordance with the working distance and field of view. To minimize the uncertainties as much as possible, the zoom lens was placed on an optical platform rather than a routinely used tripod. If a tripod is used, the tight connection between the zoom lens and the tripod may loosen over a long time period. As a result, the zoom lens may suffer from creep deformation, which will also induce errors into measurement results. The support of the platform is made of invariable alloy. With a thermal expansion coefficient of 1.6 × 10⁻⁶/^oC and a height of 0.80 m, the thermal expansion of the platform induced by air temperature variation can therefore be ignored. The monitoring target is LED so as to provide night-time visibility in long-term continuous monitoring. A total of five LEDs were fixed on a target panel of 500 × 500 mm² in size to examine the repeatability of the testing results as well as the extent of lens distortion. The target panel was mounted on a support made of invariable alloy as well. Similarly, the thermal expansion of the support due to the air temperature variation can also be ignored. In addition, to determine the displacement measurement error, the true values of the displacement of the monitoring target have to be known a priori. To simplify the testing, the monitoring target was kept fixed in the tests, whose real displacements could therefore be thought of as zeros. Correspondingly, the displacement detected by the vision measurement system is namely the displacement measurement error. Furthermore, the vision measurement system and the monitoring targets, which were some 80 m apart, were placed on the ground floor of the single-story laboratory. It is therefore reasonable to maintain that the effect of the temperature-induced deformation of the building on the testing results can be considered negligible. The optical axis of digital camera was tuned perpendicular to the object plane to the utmost to ensure the parallelism between image plane and object plane. By doing so, the object is equally scaled down into the image plane, thereby the conversion between the displacement in the image plane and the counterpart in the object plane can be achieved with a simple scale factor approach (Feng and Feng, 2017; Wahbeh et al., 2003; Wu and Casciati, 2014). The scale factor was determined by making use of the known distance between either two LEDs in the target panel, which can then be used for the conversion between image coordinates and space coordinates. After the completion of calibration, the digital camera worked uninterruptedly at a frame rate of 1 fps.

Figure 1.

Experimental setup.

In the indoor environments, it is logical to infer that the indoor air temperature is the main source of displacement measurement error. Therefore, comprehensive monitoring of temperature, including temperatures of digital camera and zoom lens, as well as indoor air temperature, was implemented. The layout of temperature sensors on the vision measurement system is also shown in Figure 1. The digital camera was equipped with four temperature sensors with one on each of the four side surfaces. Three temperature sensors were attached on the top surface of zoom lens, distributing along its longitudinal axis. One temperature sensor was employed to monitor the indoor air temperature. As a result, a total of eight temperature sensors were installed in the tests. The resolution of the temperature sensor is 0.1°C and the sampling period is 1 min. The main reason behind the use of 1 min temperature is that temperature usually experiences mild variations which may be thought to be constant in 1 min. In addition, the use of 1 min average also helps remove random noises in displacement measurement data. To make the vision measurement system and temperature measurement system operate in synchronization, the clocks of the two systems were synchronized with the Internet time server.

Experimental results

Temperature measurement data

Several sets of measurement data have been obtained from the videogrammetric measurement tests. Among them, Dataset 1 has the largest amount of measurement data. Therefore, it was most deeply analyzed in this study. In this dataset, a total of 1713-h continuous measurement data were obtained from a test that commenced on 21 January 2017 and terminated on 2 April 2017. Figure 2 displays the temperatures of digital camera, temperatures of zoom lens, and indoor air temperature in Dataset 1. It is worth noting that the temperatures do show daily fluctuation patterns though they are somewhat submerged in Figure 2 because of the long time scale. In the indoor environments, the air temperature fluctuated in the range of 6.4°C–21.1°C over this period. Consequently, the temperatures of vision measurement system, including those of digital camera and zoom lens, generally fluctuated in step with the indoor air temperature. Nevertheless, the temperatures of digital camera were much larger than the indoor air temperature because it was a heating element itself. To quantify the degree of correlation between indoor air temperature and temperatures of vision measurement system, the correlation coefficients between them were calculated. The correlation coefficient R is defined as

R = \frac{S_{xy}}{\sqrt{S_{xx}} \cdot \sqrt{S_{yy}}}

(1)

where $S_{xy}$ is the covariance between vectors x and y; $S_{xx}$ and $S_{yy}$ are the variance of vector x and y, respectively. Table 1 summarizes the correlation coefficients between indoor air temperature and temperatures of vision measurement system. All of them are more than 0.990, which demonstrates that the temperatures of vision measurement system completely correlated with the indoor air temperature. It is worth noting that some time lags between temperatures of vision measurement system and indoor air temperature have been observed. To estimate the time lag, the time axis of temperature of vision measurement system is shifted relative to that of ambient temperature, resulting in a series of time-shifted sequence for temperature of vision measurement system. Then, the correlation coefficient is calculated for each time-shifted sequence. After that, the time lag is identified as the one that corresponds to the maximum correlation coefficient. To save space, only the correlation coefficient that corresponds to the optimal time lag is presented in Table 1.

Figure 2.

Temperatures of vision measurement system and indoor air temperature in Dataset 1: (a) indoor air temperature, (b) temperatures of digital camera, and (c) temperatures of zoom lens.

Table 1.

Correlation coefficients between temperatures of vision measurement system and indoor air temperature in Dataset 1.

Temperatures of digital camera				Temperatures of zoom lens
Top	Bottom	Left	Right	Fore-end	Mid-section	Rear-end
0.993	0.994	0.994	0.994	0.998	0.999	0.998

Displacement measurement data

No notable differences were observed between the displacements of the five targets. With reference to the displacement of the central target, the maximum difference in average is 0.09 and 0.18 pixels in the horizontal and vertical directions, respectively. Therefore, the lens distortion can be considered trivial at each individual time instant. Hereafter, the displacement of the central target is used as the representative unless otherwise specified. Figure 3 plots the displacement of the central target. The displacement was averaged every 1 min and was deducted by the mean displacement at 0 o’clock sharp on the first day of test (zero reference). As the target panel was kept fixed in the test and the thermal expansion of its support was ignorable, the true values of the displacement of the monitoring target were thought of as zeros. Meanwhile, the thermal expansion of the optical platform, where the vision measurement system was placed, was also negligible. Therefore, it is reasonable to maintain that the recorded displacement is exactly the measurement error. As it is seen, notable measurement errors are detected in both horizontal and vertical displacements. Comparatively speaking, the measurement error in the vertical direction is much more significant than its counterpart in the horizontal direction. The ranges of the horizontal and vertical measurement errors are 8.64 (–2.93 to 5.71) pixels and 46.20 (–3.05 to 43.15) pixels, respectively. Roughly speaking, the measurement error in the vertical direction is nearly five times the counterpart in the horizontal direction. Assume that the vision measurement system is set to have a field of view of 1000 × 1000 mm² parallel to the image plane, which corresponds to a scale factor of 0.977 mm/pixel, the displacement measurement errors in the horizontal and vertical directions are 8.438 and 45.117 mm, correspondingly. Recalling that the indoor air temperature in this period changed by 14.7°C only, the displacement measurement error, especially the vertical one, would become intolerable in long-term continuous monitoring under outdoor environments.

Figure 3.

Displacement of central target in Dataset 1: (a) horizontal and (b) vertical.

It is also observed that the displacement measurement errors present daily fluctuation patterns. In terms of a long time scale, the daily fluctuation pattern is detectable in the horizontal measurement error, while it is unapparent in the vertical measurement error. Figure 4 zooms in the displacement measurement errors and indoor air temperature from Day 26 to Day 29. Obvious daily fluctuation patterns are detected in both horizontal and vertical measurement errors. In terms of the horizontal measurement error, it varies almost in step with the indoor air temperature. Namely, it increases as the temperature rises and decreases as the temperature drops. As a result, its daily fluctuation pattern is nearly the same as that of the indoor air temperature. As mentioned earlier, not only the support of the target panel but also the support of the vision measurement system has negligible thermal expansion. It is therefore concluded that the horizontal measurement error is an outcome mainly attributed to the thermal actions of the vision measurement system. As far as the vertical measurement error is concerned, it does present a daily fluctuation pattern as well. Nevertheless, its daily fluctuation pattern is much more complicated. Roughly speaking, it decreases as the temperature rises around 7 am to 15 pm. Then, it rebounds rapidly, usually in not more than 4 h. After that, it decreases again as the temperature drops till 7 am the second day. In other words, the rise in temperature may lead to either the decrease or the increase in the vertical measurement error and vice versa. Though the daily fluctuation pattern of the vertical measurement error differs significantly from that of the indoor air temperature, it may also be attributed to an outcome of the thermal actions of the vision measurement system as both of them have the same period of 24 h. In this sense, it seems that the effects of temperature variation on the vertical measurement error are complicated. As it was stated in previous studies (Handel, 2009; Yu et al., 2014), not only the thermal expansion of the mechanical components of a digital camera but also the temperature-induced change in the refractive index of the lens optical material contribute to the image drift, which possibly makes the effects of temperature variation on the vertical measurement error complex and intricate.

Figure 4.

Measurement errors and indoor air temperature from Day 26 to Day 29 in Dataset 1: (a) horizontal and (b) vertical.

In addition to the periodic component (daily fluctuation), it is obvious that the measurement error also contains a trend component. Recalling Figure 2, it is found that the consistency in the overall trends between vertical measurement error and indoor air temperature is much superior to the counterpart between horizontal measurement error and indoor air temperature. To amplify this, Figure 5 presents the time history of indoor air temperature and that of measurement error in the same plot. As it is seen, the vertical measurement error coincided well with the indoor air temperature in terms of the overall trend. It therefore substantiates that the vertical measurement error is an outcome mainly attributed to the thermal actions of the vision measurement system. In the first 42 days, in which the trend component of the indoor air temperature maintained almost stationary, the variation of horizontal measurement error also coincided well with the change in indoor air temperature, thanks to the perfect consistency in their daily fluctuation patterns. After Day 42, the trend component of the indoor air temperature presented an overall ascending trend. Nevertheless, the horizontal measurement error in this period showed no obvious trend except for the daily fluctuation pattern. As a result, the horizontal measurement error commenced to deviate far from the indoor air temperature after Day 42.

Figure 5.

Measurement errors and indoor air temperature in Dataset 1: (a) horizontal and (b) vertical.

Correlations between measurement errors and temperatures

As an illustration, Figure 6 plots the displacement measurement errors versus the indoor air temperature in Dataset 1. As it is seen in Figure 6(a), the curve can be divided into three segments. In the first segment, the horizontal measurement error increases proportionally to the indoor air temperature. It is mainly attributed to the perfect consistency in their daily fluctuation patterns. At the temperature around 11°C, a turning point appears in the curve, where it begins to deviate slightly from the linear relationship accompanied by a reduction in the inclination, pronouncing the beginning of the second segment. As the temperature further increases, the nonlinearity in their relationship gets more apparent and the inclination of the curve becomes smaller. At the temperature around 15°C, a second turning point occurs, where the inclination of the curve approaches to zero, indicating the termination of the second segment. After that, the horizontal measurement error fluctuates in a small range only regardless of the significant increase in the indoor air temperature, which characterizes the third segment. It is mainly attributed to the fact that the fluctuation of horizontal measurement error is dominated by the periodic component rather than the trend component in the indoor air temperature. In contrast, the vertical measurement error and indoor air temperature generally conforms to a favorable linear relationship in the whole temperature range. Recalling the satisfactory consistency in the overall trends between vertical measurement error and indoor air temperature, it is therefore concluded that the trend component of the temperature variation dominates the variation of the vertical measurement error. Meanwhile, the complicated daily fluctuation pattern of the vertical measurement error increases the discreteness. The relationships between displacement measurement errors and temperatures of vision measurement system are generally similar. To save space, the scatter plots between them are not presented. What is particularly worth mentioning is that turning points are also found in the relationships between horizontal measurement error and temperatures of vision measurement system.

Figure 6.

Displacement measurement error versus indoor air temperature in Dataset 1: (a) horizontal and (b) vertical.

To further validate the relationship between horizontal measurement error and indoor air temperature revealed in Dataset 1, another set of measurement data, namely, Dataset 2, is also explored. The Dataset 2 was acquired from 2 December 2016 to 11 January 2017, resulting in a total of 977-h measurement data. Figure 7 presents the horizontal measurement error versus the indoor air temperature. In Dataset 2, the indoor air temperature that corresponds to the zero reference of the horizontal measurement error is 13.4°C, while it is 9.4°C in Dataset 1. As a result, the minimum horizontal measurement error in Dataset 2 is smaller than that in Dataset 1 (–5.84 vs −2.93 pixels). It is seen that the curve in Dataset 2 can also be divided into three segments. Generally, the two turning points are also roughly located around the temperature of 11°C and 15°C, respectively. As the indoor air temperature exceeds 15°C, the horizontal measurement error does tend to remain constant with small fluctuations, which is in line with expectations. Recalling Figure 6(a), it is found that the horizontal segment in Dataset 2 is not as prominent as that in Dataset 1. It is mainly attributed to the small amount of the measurement data acquired at the indoor air temperature exceeding 15°C. To further evidence the presence of the horizontal segment in the relationship between horizontal measurement error and indoor air temperature, Dataset 3 that was obtained at the indoor air temperatures far beyond 15°C is presented. Figure 8 shows a continuous 3-day segment of the horizontal measurement error and indoor air temperature. In this period, the indoor air temperature varied between 23.3°C and 26.8°C, which was far beyond 15°C. On the first day, the indoor air temperature showed an obvious daily fluctuation pattern. As expected, the horizontal measurement error also presented a prominent daily fluctuation pattern. On the succeeding 2 days, however, the indoor air temperature dropped almost monotonously without the presence of a notable daily fluctuation pattern. Meanwhile, the horizontal measurement error maintained almost constant with small fluctuations only. It therefore consolidates the presence of a horizontal segment in the relationship between horizontal measurement error and indoor air temperature when the latter exceeds a certain value.

Figure 7.

Horizontal measurement error versus indoor air temperature in Dataset 2.

Figure 8.

A continuous 3-day segment of horizontal measurement error and indoor air temperature in Dataset 3.

To characterize the linear relationships between displacement measurement errors and temperatures, linear regression analyses between them were carried out to formulate their correlation models. As the horizontal measurement error and temperatures present an obvious nonlinear relationship, the linear regression analyses between them were not conducted. The two regression coefficients, that is, slope $α$ and interception $β$ , were obtained by the least-squares method. In addition, correlation coefficients between them were also calculated to quantify the degree of correlation. Table 2 summarizes the regression coefficients and correlation coefficients between the vertical measurement error and temperatures of vision measurement system as well as indoor air temperature. As mentioned earlier, there exist some time lags between temperatures of vision measurement system and indoor air temperature. Accordingly, there may be some time lags between vertical measurement error and temperature as well. The time lag between them can be estimated in a similar way and only the results corresponding to the optimal time lag are shown in Table 2. The correlation coefficient varies between 0.884 and 0.912, indicating that the vertical measurement error correlated well with the temperatures of vision measurement system as well as the indoor air temperature. It therefore consolidates that the vertical measurement error is also an outcome attributable to the thermal actions of the vision measurement system. Because the temperatures of vision measurement system are completely correlated with the indoor air temperature, as it is shown in Table 1, there are slight differences in the correlation coefficients. Generally, the correlation coefficients corresponding to the temperatures of the same component of the vision measurement system are barely distinguishable. Nevertheless, discriminable differences are observed in the correlation coefficients that correspond to the temperatures of the different components of the vision measurement system. Among them, the temperatures of zoom lens are most correlated with the vertical measurement error. The maximum correlation coefficient between them approaches to 0.912. In contrast, the temperatures of digital camera are less well correlated with the vertical measurement error. The maximum correlation coefficient between them is 0.888, which is even smaller than the correlation coefficient that corresponds to the indoor air temperature.

Table 2.

Regression coefficients and correlation coefficients for vertical measurement error in Dataset 1.

Temperature		Correlation coefficient (R)	Slope ( $α$ )	Interception ( $β$ )
Digital camera	Top	0.884	2.858	−74.013
	Bottom	0.888	2.938	−74.316
	Left	0.886	2.906	−76.195
	Right	0.887	2.908	−75.785
	Mean	0.886	2.902	−75.418
Zoom lens	Fore-end	0.911	3.347	−26.355
	Mid-section	0.912	3.364	−26.911
	Rear-end	0.912	3.391	−27.904
	Mean	0.912	3.367	−27.052
Indoor air		0.903	3.327	−24.859

Wavelet decomposition of measurement data

As shown above, the temperature and displacement measurement error contain not only periodic component but also trend component; it is desirable to separate the overall trend from the daily fluctuation pattern and then scrutinize their characteristics individually. The wavelet transform, which is capable of providing a time–frequency representation of a time domain signal, is one of the techniques fit for this purpose. It is a tool that cuts up data, functions, or operators into different frequency components and studies each component with a resolution matched to its scale (Daubechies, 1992). The data with coarse resolution contain information about low-frequency components and data with fine resolution contain information about high-frequency components. It has been used in many applications because of its effectiveness in detecting facets in signals. This study also employed the wavelet transform to decompose the temperatures and displacement measurement errors into overall trend and daily fluctuation pattern. The continuous wavelet transform is found by calculating the convolution of a time domain signal $x (t)$ , and scaled and shifted versions of a mother wavelet function $ψ (t)$ , that is

C (a, b) = \frac{1}{\sqrt{a}} \int_{- \infty}^{+ \infty} x (t) ψ (\frac{t - b}{a}) dt = \int_{- \infty}^{+ \infty} x (t) ψ_{a, b} (t) dt

(2)

where a is a scale factor determining the dilation of mother wavelet; b is a translation factor associated with the translation of mother wavelet; and $C (a, b)$ is the wavelet coefficient which is a function of scale factor and translation factor. The discrete wavelet transform follows by setting $a = 2^{j}$ and $b = k$ , where $j, k \in Z$ , Z is the integer domain. The original signal can be reconstructed from the wavelet coefficients by inverse discrete wavelet transform, which is referred to as wavelet reconstruction.

The wavelet transform is dependent on the mother wavelet. Several types of mother wavelet exist for discrete wavelet transform, such as Daubechies, Coiflets, and Symlets. Among them, Daubechies wavelet family has been by far the most widely used (Rafiee et al., 2009), and therefore, a mother wavelet from this family was also employed in this study. Because of the lack of well-recognized criteria, the order of the mother wavelet was determined by trial-and-error method based on the intrinsic characteristics of the signal being analyzed. To do so, Daubechies wavelets with a variety of orders were tested on the indoor air temperature in Dataset 1. To save space, they are not presented. In general, the approximations of the indoor air temperature generated by low-order Daubechies wavelets are relatively rough compared to the counterparts yielded by high-order ones. As the order of the mother wavelet increases, the difference between the approximations becomes trivial. For the decomposition of the displacement measurement errors and temperatures, the order of eight was thought to be sufficient and thereby was finally used. Another key issue in the wavelet transform is the selection of the decomposition level. Similarly, it was also determined by the trial-and-error method. Making use of the mother wavelet db8, the details of the indoor air temperature at the levels 1–15 were obtained. They are not presented as well to save space. The first several levels are obviously insufficient. At the level 10 or above, no notable difference can be detected in the details of the indoor air temperature. It is therefore inferred that 10 levels are sufficient for the decomposition of the temperatures as well as displacement measurement errors and thereby was finally used.

The details of indoor air temperature as well as those of displacement measurement errors in Dataset 1 are plotted in Figure 9. The details of the temperatures of vision measurement system are not presented as they are analogous to those of indoor air temperature. By separating the details from the approximations, the daily fluctuation pattern of indoor air temperature is prominent even in a long time scale. Furthermore, the daily fluctuation pattern is apparent in not only horizontal measurement error but also vertical measurement error. It is obvious that the daily fluctuation pattern of horizontal measurement error coincides well with that of indoor air temperature. Nevertheless, it seems that the daily fluctuation pattern of vertical measurement error does not coincide well with that of indoor air temperature. To evidence this, Figure 10 shows the details of displacement measurement error versus those of indoor air temperature. As expected, the details of horizontal measurement error and those of indoor air temperature conform to a satisfactory linear relationship. Meanwhile, a nonlinear relationship between the details of vertical measurement error and those of indoor air temperature is obvious. To quantify the degree of correlation, Table 3 summarizes the correlation coefficients between the details of horizontal measurement error and those of the temperatures of vision measurement system as well as indoor air temperature. Similarly, the results that correspond to the optimal time lag are presented. Generally speaking, the details of all the temperatures are well correlated with those of horizontal measurement error as all the correlation coefficients are greater than 0.840. Comparatively speaking, the correlations between the details of the temperatures of zoom lens and horizontal measurement error are superior to the counterparts between the details of the temperatures of digital camera and horizontal measurement error. All the correlation coefficients that correspond to the details of the temperatures of zoom lens are larger than 0.902. Nevertheless, the maximum correlation coefficient between the details of the temperatures of digital camera and those of horizontal measurement error is 0.851, which is even less than the correlation coefficient that corresponds to the details of indoor air temperature.

Figure 9.

Details of indoor air temperature and displacement measurement errors in Dataset 1: (a) indoor air temperature, (b) horizontal displacement measurement error, and (c) vertical displacement measurement error.

Figure 10.

Details of displacement measurement error versus details of indoor air temperature in Dataset 1: (a) horizontal and (b) vertical.

Table 3.

Correlation coefficients between details of horizontal measurement error and temperatures in Dataset 1.

Temperature of digital camera					Temperature of zoom lens				Indoor air temperature
Top	Bottom	Left	Right	Mean	Fore-end	Mid-section	Rear-end	Mean	Indoor air temperature
0.841	0.850	0.851	0.843	0.849	0.902	0.904	0.902	0.904	0.866

Figure 11 presents the approximations of indoor air temperature as well as those of displacement measurement errors in Dataset 1. The approximations of the temperatures of vision measurement system are also analogous to those of indoor air temperature and they are therefore not presented as well. In the first 42 days, the approximations of indoor air temperature fluctuated without the presence of an obvious overall trend. After that, an overall ascending trend was apparent. The approximations of horizontal measurement error generally coincided with those of indoor air temperature in the first 42 days and deviated significantly from those of indoor air temperature in the last 30 days. Meanwhile, the approximations of vertical measurement error coincided well with those of indoor air temperature in all the days. To further evidence this, Table 4 lists the correlation coefficients between the approximations of vertical measurement error and those of the temperatures of vision measurement system as well as indoor air temperature. Again, the results corresponding to the optimal time lag are shown. As it is seen, all the correlation coefficients are greater than 0.910, which consolidates that they are consistent. Again, the approximations of the temperatures of zoom lens are most correlated with those of vertical measurement error. All the correlation coefficients corresponding to the approximations of the temperatures of zoom lens are larger than 0.928. In contrast, the approximations of the temperatures of digital camera are less well correlated with those of vertical measurement error. The maximum correlation coefficient between them is 0.914, which is less than the correlation coefficient that corresponds to the approximations of indoor air temperature. With reference to the correlation coefficients between undecomposed temperatures and vertical measurement error, as shown in Table 2, the correlation coefficients between their approximations are larger. It is therefore suspected that the correlation between the details of vertical measurement error and temperatures degrade the degree of correlation between them as a whole.

Figure 11.

Approximations of indoor air temperature and displacement measurement errors in Dataset 1: (a) horizontal and (b) vertical.

Table 4.

Correlation coefficients between approximations of vertical measurement error and temperatures in Dataset 1.

Temperature of digital camera					Temperature of zoom lens				Indoor air temperature
Top	Bottom	Left	Right	Mean	Fore-end	Mid-section	Rear-end	Mean	Indoor air temperature
0.913	0.913	0.914	0.914	0.914	0.928	0.929	0.929	0.929	0.927

Discussions

The relationships between the measurement errors and the temperatures of the vision measurement system revealed in this study may have significant implications in the deformation monitoring of civil engineering structures. In terms of horizontal measurement error, one of the most interesting findings is that it fluctuates within a small range when the temperatures of vision measurement system exceed a certain value. Recalling Dataset 1, when the minimum indoor air temperature exceeded 11.9°C in the last 30 days, the horizontal measurement error fluctuated in a range of 1.97–5.71 pixels. Assuming a field of view of 1000 × 1000 mm² parallel to the image plane again, the horizontal measurement error in the absolute sense is 3.65 mm, correspondingly. At present, the state-of-the-art digital camera can have a much higher resolution. Furthermore, the field of view is usually smaller in the field monitoring. Thereby, the horizontal measurement error may be further depressed. On these grounds, it is therefore logical to speculate that the horizontal measurement error may be confined in an acceptable range if the temperatures of vision measurement system can be maintained beyond a certain value. As might be imagined, it will be much easier to maintain the temperatures of vision measurement system beyond a certain value than to fix them at a constant value, it therefore becomes practical to restrict the temperature-induced horizontal measurement error of the videogrammetry within an acceptable range in the deformation monitoring of civil engineering structures. To this end, the minimum temperature above which the horizontal measurement error can be confined to small fluctuations has to be found. In this study, the indoor air temperature corresponding to this is roughly estimated as 15°C. Though it may not be necessary, a more accurate estimation may be obtained by fitting a regression curve to the measurement data and/or by performing videogrammetric measurement test in accurately temperature-controlled environments. In addition, measures to maintain the temperatures of vision measurement system should be worked out. These issues will be carried out in the future work.

Recognizing the satisfactory linear relationships between vertical measurement error and temperatures of vision measurement system as well as indoor air temperature, a direct intuition suggests formulating linear correlation models between them to predict the vertical measurement error. Nevertheless, it is also noted that the rise in temperature may lead to either the decrease or the increase in the vertical measurement error and vice versa, which may make the reproduced or predicted value far deviate from the measured value. As a result, the linear correlation models between vertical measurement error and temperatures of vision measurement system could not be used for prediction purpose. Another possible way to subtract the vertical measurement error from the measured total vertical displacement is a time–frequency domain approach, recognizing that the temperature in ambient environments experiences extremely low-frequency variations. To do so, one of the key issues to be concerned is the deformation induced by the thermal expansion of a civil engineering structure itself. Long-span bridges and high-rise buildings are two kinds of structures that are most frequently monitored in reality. In the deformation monitoring of high-rise buildings, the in-plane and out-of-plane horizontal displacements are the major concerns, while the in-plane vertical and horizontal displacements are the main focuses in the deformation monitoring of long-span bridges. As mentioned earlier, the temperature-induced horizontal measurement error may be confined in an acceptable range if the temperatures of vision measurement system can be maintained beyond a certain value, leaving the separation of the temperature-induced vertical measurement error from the measured total vertical displacement of a long-span bridge that still needs to be worked out. To this end, the magnitude of the thermal expansion of a long-span bridge along the vertical direction is examined. The coefficient of thermal expansion of concrete or steel is in the order of 10⁻⁵/^oC. Meanwhile, the girder of a long-span bridge usually has a height less than 4 m. For example, the steel box girder of the Jiangyin Bridge, which has a main span of 1385 m, is 3 m in height (Zhou et al., 2013). Assuming an extreme temperature of 50°C and a symmetric thermal expansion with reference to the midline of the cross section of a bridge deck, it can be optimistically estimated that the thermal expansion of a long-span bridge in the vertical direction will be within 1 mm. As a result, the displacement component induced by the thermal expansion of a long-span bridge can be ignored in the total vertical displacement. Subsequently, the temperature-induced vertical measurement error can be subtracted from the measured total vertical displacement by making use of a time–frequency domain approach such as the wavelet decomposition.

Conclusion

The performance of the videogrammetric displacement monitoring technique under varying ambient temperature has been investigated in this study. Videogram-metric measurement tests, which last intermittently for more than half a year, have been carried out in indoor environments. Making use of the long-term measurement data, the characteristics of the temperature-induced measurement errors have been obtained by examining them as a whole as well as their wavelet decomposed components. Correlation analyses have been carried out to characterize the relationships between displacement measurement error and temperatures of vision measurement system. Making use of the unique features of the temperature-induced measurement error, the possible means to eliminate the error have been proposed. The following conclusions are drawn from this study:

Temperature variation causes significant displacement measurement error. With a change of 14.7°C in the indoor air temperature, the measurement error approaches to 8.64 and 46.20 pixels in the horizontal and vertical directions, respectively. In the long-term continuous deformation monitoring of a civil engineering structure in ambient environments, the temperature-induced measurement errors may become intolerable, especially the vertical one.

In the horizontal measurement error, the periodic component is more prominent than the trend component. Its daily fluctuation pattern coincides well with the counterpart of temperature. In contrast, their trend components are almost uncorrelated with each other, especially when the temperature experiences a significant ascending/descending trend. As a result, the horizontal measurement error tends to remain constant with small fluctuations when the temperature exceeds a certain value.

In the vertical measurement error, the trend component is more outstanding than the periodic component. Its daily fluctuation pattern is complex and does not coincide well with that of temperature. However, their trend components are in satisfactory consistency. As a whole, the vertical measurement error and temperature conform to a favorable linear relationship.

Among all the temperatures, the temperature of zoom lens is most correlated with the displacement measurement error. The details of temperatures of zoom lens are most correlated with those of horizontal measurement error. The correlation coefficient between them reaches to 0.904. Similarly, the approximations of temperatures of zoom lens are most correlated with those of vertical measurement error. The correlation coefficient between them approaches up to 0.929.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by grants from the National Natural Science Foundation of China (project nos 51578424 and 51208384), Science Technology Department of Zhejiang Province, China (project no. 2012R10071), Wenzhou Science and Technology Bureau, China (project no. S20150018), Research Grants Council of Hong Kong SAR (Grant No. PolyU 152241/15E), and Zhejiang Provincial National Science Foundation of China (project no. LY12E08009).

ORCID iD

HF Zhou

References

Baqersad

Poozesh

Niezrecki

(2017) Photogrammetry and optical methods in structural dynamics—a review. Mechanical Systems and Signal Processing 86: 17–34.

Busca

Cigafa

Mazzoleni

(2014) Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system. Experimental Mechanics 54: 255–271.

Chang

Xiao

(2010) An integrated visual-inertial technique for structural displacement and velocity measurement. Smart Structures and Systems 6: 1025–1039.

Daubechies

(1992) Ten Lectures on Wavelets. Philadelphia, PA: Society for Industrial and Applied Mathematics.

Feng

(2016) Vision-based multipoint displacement measurement for structural health monitoring. Structural Control and Health Monitoring 23: 876–890.

Feng

(2017) Experimental validation of cost-effective vision-based structural health monitoring. Mechanical Systems and Signal Processing 88: 199–211.

Handel

(2009) Analyzing the influences of camera warm-up effects on image acquisition. IPSJ Transactions on Computer Vision and Applications 1: 12–20.

Jiang

Jáuregui

White

(2008) Close-range photogrammetry applications in bridge measurement: literature review. Measurement 41: 823–834.

Lee

Fukuda

Shinozuka

(2007) Development and application of a vision-based displacement measurement system for structural health monitoring of civil structures. Smart Structures and Systems 3: 373–384.

10.

Liu

Burner

Jones

(2012) Photogrammetric techniques for aerospace applications. Progress in Aerospace Sciences 54: 1–58.

11.

Cui

Teng

(2017) Structural displacement and strain monitoring based on the edge detection operator. Advances in Structural Engineering 20: 191–201.

12.

Olaszek

(1999) Investigation of the dynamic characteristic of bridge structures using a computer vision method. Measurement 25: 227–236.

13.

Park

Lee

Jung

(2010) Vision-based displacement measurement method for high-rise building structures using partitioning approach. NDT & E International 43: 642–647.

14.

Park

Kim

(2015) 3D displacement measurement model for health monitoring of structures using a motion capture system. Measurement 59: 352–362.

15.

Podbreznik

Potočnik

(2012) Assessing the influence of temperature variations on the geometrical properties of a low-cost calibrated camera system by using computer vision procedures. Machine Vision and Applications 23: 953–966.

16.

Rafiee

Tse

Harifi

(2009) A novel technique for selecting mother wavelet function using an intelligent fault diagnosis system. Expert Systems with Applications 36: 4862–4875.

17.

Robson

Clarke

Chen

(1993) The suitability of the Pulnix TM6CN CCD camera for photogrammetric measurement. In: Proceedings of SPIE, videometrics II, vol. 2067 (ed El-Hakim

), Boston, MA, 22 October. Bellingham, WA: SPIE Press.

18.

Sharpe

(2008) Handbook of Experimental Solid Mechanics. New York: Springer.

19.

Wahbeh

Caffrey

Masri

(2003) A vision-based approach for the direct measurement of displacements in vibrating systems. Smart Materials and Structures 12: 785–794.

20.

Widenhorn

Blouke

Weber

(2002) Temperature dependence of dark current in a CCD. In: Proceedings of SPIE, sensors and camera systems for scientific, industrial, and digital photography applications III, vol. 4669 (eds Sampat

Blouke

Canosa

), San Jose, CA, 24 April, pp. 193–201. Bellingham, WA: SPIE Press.

21.

Casciati

(2014) Local positioning systems versus structural monitoring: a review. Structural Control and Health Monitoring 21: 1209–1221.

22.

Chao

Jiang

(2014) The effects of temperature variation on videometric measurement and a compensation method. Image and Vision Computing 32: 1021–1029.

23.

Zhou

(2013) Structural health monitoring of the Jiangyin Bridge: system upgrade and data analysis. Smart Structures and Systems 11: 637–662.

24.

Zhou

Zheng

Xie

(2017) Temperature effects on vision measurement system in long-term continuous monitoring of displacement. Renewable Energy 114: 968–983.