A Flexible Calibration Approach for Cameras with Double-Sided Telecentric Lenses

Abstract

With their constant perspective and large magnification in the working distance, double-sided telecentric lenses have been widely used in machine-vision applications. This paper puts forward a flexible calibration approach for the double-sided telecentric camera. Based on an orthographic projection model considering the major sources of lens distortions, a two-step calibration procedure is proposed. In this approach, the camera parameters apart from the lens distortions are achieved by a closed-form solution. Then a double non-linear optimization is performed to refine all the parameters, including the distortion coefficients and distortion centre. In addition, to achieve a flexible calibration procedure, the calibration pattern used is a cheap print product rather than a professional customized calibration pattern. Simulation and real-world experiments are performed to validate the performance of the proposed calibration approach. In addition, the comparison experiments between the print calibration pattern and professional calibration pattern are carried out, and the accuracy of calibration results are at the same level.

Keywords

Camera Calibration Double-sided Telecentric Lens Parallel Projection Distortion Homographic Matrix

1 Introduction

Double-sided telecentric lenses have been widely used in many machine-vision applications such as noncontact measurement and inspection systems [1–3]. The principal scheme of a double-sided telecentric lens is shown in Figure 1. Unlike the perspective projection of the pinhole camera model, double-sided telecentric lenses provide purely orthographic projections of scene points by locating an aperture stop at the focal point of a lens [4]. Thus, these lenses provide constant perspective and a large magnification depth of field. This property makes it easier to measure or compare physical lengths of objects independently from the depths of the objects from the camera [5]. For this reason, they are often well suited for gauging and a variety of other demanding machine vision applications [6–8].

Figure 1.

The schematic diagram of light path in double-sided telecentric lenses

Generally, camera calibration is an essential part of metric information [9, 10]. Therefore, the calibration of a telecentric camera is essential for its optical measurement application. The camera calibration for a pinhole model is widely discussed due to its important role in optical measurement, and many calibration methods have been proposed. Tsai proposes a two-step calibration technique considering radial lens distortion [11]. Weng presents a camera model with three types of distortion in the calibration technology [12]. The flexible camera calibration method based on a 2D pattern is proposed by Zhang [13]. There are also calibration methods based on the optical method [14], camera motion [15], structure light [16], and even a star pattern [17]. However, these calibration methods are based on perspective projection in the pinhole camera model, which is not available for telecentric camera calibration.

For telecentric camera calibration, Zhu provides a linear fitting calibration method [18], which is able to achieve the intrinsic parameters for the distortion-free telecentric camera. Li develops a two-step method to calibrate telecentric lenses without considering the ambiguity of extrinsic parameters [19]. The experimental results show that thin-prism distortion should not be ignored, which differs in a traditional perspective camera. Chen proposed an effective calibration method for a telecentric camera in an object space, considering lens distortion [20]. In this solution, the problem of sign ambiguity from the planar object-based calibration technique is successfully solved. However, the four elements of the truncated extrinsic attitude matrices of the camera have been adopted in the non-linear refinement, instead of the independent variable of attitude description. Further, it is assumed that the centre of distortion is known as the image centre in the above literature, but telecentric lenses perform parallel projection and there is no projection centre; however, this is not a safe assumption in general [21]. Experiments carried out by Richard Hartley demonstrated that the centre of distortion may be significantly displaced from the centre of the image, or the principal point of the camera [22].

A flexible camera calibration method for telecentric lenses is proposed in this paper. In this method, a cheap print product replaces the professional customized calibration pattern to achieve a flexible calibration procedure. With the pattern and corresponding algorithm, telecentric camera parameters are all calibrated, including the rotation matrix, translation vector, effective magnification, lens distortion and distortion centre. In addition, the independent variables of attitude description by Rodrigues's formula have been adopted in the non-linear refinement process. The concrete method is as follows:

Based on the orthographic projection model, the calibration procedure is divided into two steps. In the first, the homographic matrix estimated from the imaging equation is used to obtain an initial value of camera parameters with the distortion-free telecentric camera model. Then, a planar pattern with an additional stage is used to avoid the ambiguity of extrinsic parameters. In the second step, considering the accuracy of calibration pattern used in the work, a non-linear optimization is carried out first to obtain more reliable data for the calibration pattern. Then another non-linear optimization is applied for all parameter estimations including the lens distortion and its centre. As the optimization is carried out for all parameters, a refined result is achieved.

The rest of the paper is organized as follows. The double-sided telecentric camera model, including distortion-free telecentric camera model and lens distortion model, is presented in Section 2. The calibration theory and method in detail are in Section 3. Section 4 provides results of computer simulation and real-world experiments, and the comparison experiments are also presented. Section 5 is the conclusion of this paper.

2 Double-sided Telecentric Camera Model

2.1 Distortion-free telecentric camera model

The imaging model of a camera with a double-sided telecentric lens can be illustrated by Figure 2. (x_w, y_w, z_w) and (x_c, y_c, z_c) are the 3D coordinates of the object point P in the 3D world coordinate system and camera coordinate system, respectively. (u, v) is the image coordinate of P(x_c, y_c, z_c) in pixels. The projection of an arbitrary point P to the undistorted image plane in pixel units is expressed as follows [19]:

[\begin{matrix} u \\ v \\ 1 \end{matrix}] = [\begin{matrix} M / d_{u} & 0 & 0 & u_{0} \\ 0 & M / d_{v} & 0 & v_{0} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{c} \\ y_{c} \\ z_{c} \\ 1 \end{matrix}],

(1)

where M is the effective magnification of the double-sided telecentric lens, and (u₀, v₀) is the coordinate of the image plane centre. d_u and d_v denote the pixel pitches in u and v directions, respectively.

Suppose that R and t are the rotation matrix and translation vector respectively, which relate the world coordinate system to the camera coordinate system, and the relationship between the world and camera coordinate systems can be described as:

[\begin{matrix} x_{c} \\ y_{c} \\ z_{c} \\ 1 \end{matrix}] = [\begin{matrix} R & t \\ 0_{1 \times 3} & 1 \end{matrix}] [\begin{matrix} x_{w} \\ y_{w} \\ z_{w} \\ 1 \end{matrix}] = [\begin{matrix} r_{11} & r_{12} & r_{13} & t_{x} \\ r_{21} & r_{22} & r_{23} & t_{y} \\ r_{31} & r_{32} & r_{33} & t_{z} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{w} \\ y_{w} \\ z_{w} \\ 1 \end{matrix}] .

(2)

Combining Eq. (1) and Eq. (2), the orthographic projection of double-sided telecentric lenses is expressed by the equation:

[\begin{matrix} u \\ v \\ 1 \end{matrix}] = [\begin{matrix} M / d_{u} & 0 & u_{0} \\ 0 & M / d_{v} & v_{0} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} r_{11} & r_{12} & r_{13} & t_{x} \\ r_{21} & r_{22} & r_{23} & t_{y} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{w} \\ y_{w} \\ z_{w} \\ 1 \end{matrix}] .

(3)

Unlike the pinhole camera model, there is no arbitrary scale factor on the left side of Eq. (3) due to the orthographic projection of the double-sided telecentric lens. The detailed derivation process of Eq. (1)–(3) is described in reference [19].

Figure 2.

The schematic diagram of imaging with double-sided telecentric lens camera

2.2 Lens distortion

As a result of several types of imperfections in design and assembly, lens distortions are inevitable and the expressions in Eq. (3) do not always hold true. There has already been research on the physical nature of the distortion of optical systems [23, 24], which depends on the distance of the object from the optical system. Although the distortions of double-sided telecentric lenses are small compared to those of other lenses, accurately identifying the distortion model is also absolutely necessary for telecentric lenses with high measuring accuracy [20].

Similar to an ordinary lens modelled by pinhole perspective projection, the distortions in the telecentric lens can be divided into radial and tangential distortions. Hence, according to Brown's model [25] distortions are given by:

[\begin{matrix} δ_{u} \\ δ_{v} \end{matrix}] = [\begin{matrix} u^{'} \\ v^{'} \end{matrix}] - [\begin{matrix} u \\ v \end{matrix}] = [\begin{array}{l} (k_{1} r^{2} + k_{2} r^{4} + k_{5} r^{6}) u + 2 k_{4} u^{2} + 2 k_{3} u v + k_{4} r^{2} \\ (k_{1} r^{2} + k_{2} r^{4} + k_{5} r^{6}) v + 2 k_{3} v^{2} + 2 k_{4} u v + k_{3} r^{2} \end{array}],

(4)

where u and v are the non-observable and distortion-free image coordinates, while u' and v' are the observable coordinates with distortion, and $r = \sqrt{u^{2} + v^{2}}$ is the radial value. In Eq. (4), k₁ ∼ k₅ are distortion parameters. There are three parameters (k_y, k₂, k₅) for radial distortion, which contain different degrees. The other two parameters (k₃, k₄) are tangential distortion parameters.

Generally, there are 11 calibration parameters in the double-sided telecentric camera models presented; five are for extrinsic parameters (the Euler angles yaw, pitch, and tilt for rotation R, the two components t₁, t₂ for the translation vector t ), and the other six are for intrinsic parameters (effective magnification M, lens distortion coefficients k₁ ∼ k₅). As mentioned in the introduction, the centre of distortion (u₀, v₀) should be taken into account in the calibration as well as in high accuracy calibration.

3 Single Camera Calibration

Inspired by Zhang [13], a planar board with control points is used to achieve the intrinsic and extrinsic parameters of the camera in this section. To allow precise calibration, the pattern should be planar within a certain degree in the world coordinate system [26].

3.1 Estimation of the homographic matrix

Without a loss of generality, we assume the calibration pattern lies on Z = 0 of the world coordinate system. Therefore, as Eq. (3) can be written as:

[\begin{matrix} u \\ v \\ 1 \end{matrix}] = [\begin{matrix} M / d_{u} & 0 & u_{0} \\ 0 & M / d_{v} & v_{0} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} r_{11} & r_{12} & t_{x} \\ r_{21} & r_{22} & t_{y} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{w} \\ y_{w} \\ 1 \end{matrix}] = K [\begin{matrix} R_{s} & t_{s} \\ 0_{1 \times 2} & 1 \end{matrix}] [\begin{matrix} x_{w} \\ y_{w} \\ 1 \end{matrix}] = H [\begin{matrix} x_{w} \\ y_{w} \\ 1 \end{matrix}] .

(5)

Suppose that h_ij, is the i^th row and j^th column element of homographic matrix H. According to Eq. (5), the following equation can be achieved:

[\begin{matrix} u \\ v \end{matrix}] = [\begin{matrix} x_{w} & y_{w} & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & x_{w} & y_{w} & 1 \end{matrix}] [\begin{matrix} h_{11} \\ h_{12} \\ h_{13} \\ h_{21} \\ h_{22} \\ h_{23} \end{matrix}] .

(6)

Thus, the elements of homographic matrix H can be easily computed by using the DLT algorithm [27].

3.2 Estimation of camera parameters from the homographic matrix

With the orthogonality of the rotation matrix R, we have:

r_{11} r_{12} + r_{21} r_{22} + r_{31} r_{32} = 0.

(7)

Besides, rotation matrix R is unitary, so we can quickly obtain:

{\begin{matrix} r_{31} = \pm \sqrt{1 - r_{11}^{2} - r_{21}^{2}} \\ r_{32} = \pm \sqrt{1 - r_{12}^{2} - r_{22}^{2}} \end{matrix} .

(8)

Thus, Eq. (9) can be achieved by substituting Eq. (8) into Eq. (7):

r_{11} r_{12} + r_{21} r_{22} = \pm \sqrt{1 - r_{11}^{2} - r_{21}^{2}} \sqrt{1 - r_{12}^{2} - r_{22}^{2}} .

(9)

Squared on both sides of the Eq. (9), we have:

1 - (r_{11}^{2} + r_{12}^{2} + r_{21}^{2} + r_{22}^{2}) + {(r_{11} r_{22} - r_{12} r_{21})}^{2} = 0.

(10)

Suppose that M/d_u = M/d_v= m, combining Eq. (3) and Eq. (10), and the following equation can be achieved:

m^{4} - (h_{21}^{2} + h_{22}^{2} + h_{11}^{2} + h_{12}^{2}) m^{2} + {(h_{11} h_{22} - h_{12} h_{21})}^{2} = 0

(11)

Generally, there are two non-negative roots for Eq. (11). This ambiguous problem can be solved easily by the conditions of $r_{i j}^{2} = h_{i j}^{2} / m^{2} \leq 1$ , where i = 1, 2 and j = 1, 2.

Once m is known, the truncated translation $t_{s} = {[\begin{matrix} t_{x} & t_{y} \end{matrix}]}^{T}$ and image plane centre (u₀, v₀) are satisfied as follows:

{\begin{matrix} m t_{x} + u_{0} = h_{13} \\ m t_{y} + v_{0} = h_{23} \end{matrix} .

(12)

Obviously, (u₀, v₀) and t _s cannot be uniquely determined. The reason for this phenomenon is that the first two elements in the last row of homographic matrix H have no other values than zero, as the telecentric camera has no projection centre (lies at infinite). Thus, the location of image plane centre can be set freely for a distortion-free telecentric camera. Without a loss of generality, we use the detector plane centre to represent the image plane centre. Then, the truncated translation t_s can be determined.

On the other hand, the elements in the upper left 2×2 submatrix of the rotation matrix R for each image can be directly achieved from Eq. (3). Then, the remaining matrix elements can be obtained from Eqs. (8) and (13) since rotation matrix R is unitary and orthogonal.

r_{3} = r_{1} \times r_{2},

(13)

where r ₁, r ₂ and r ₃ are the row vectors of the matrix R. However, the signs of r₁₃ and r₂₃ in Eq. (3) are uncertain, whereas the orthogonality of R can only provide one constraint. Chen solved the problem of sign ambiguity with the help of a micro-positioning stage [20]. Similarly, we set an additional stage with a control point (x_d, y_d, z_d) on the planar board to solve this problem. Suppose that the image coordinate of the control point is (u_d, v_d), then the r₁₃ and r₂₃ are satisfied as follows:

[\begin{matrix} r_{13} \\ r_{23} \\ 0 \end{matrix}] = {[\begin{matrix} m & 0 & u_{0} \\ 0 & m & v_{0} \\ 0 & 0 & 1 \end{matrix}]}^{- 1} [\begin{matrix} u_{d} / z_{d} \\ v_{d} / z_{d} \\ 1 / z_{d} \end{matrix}] - [\begin{matrix} r_{11} & r_{12} & t_{x} \\ r_{21} & r_{22} & t_{y} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} x_{d} / z_{d} \\ y_{d} / z_{d} \\ 1 / z_{d} \end{matrix}] .

(14)

According to reference [20], r₁₃ and r₂₃ recovered from Eq. (13) may not precisely satisfy the orthogonality of R because of the noise effect, but their signs are all perfectly disambiguated.

3.3 Non-linear optimization for all parameter estimations

For a distortion-free telecentric camera, we can obtain the closed-form solution of its parameters with only one captured image from Section 3.2. However, the lens distortions are inevitable, as mentioned previously. Thus, a non-linear optimization is performed to refine the all camera parameters considering lens distortion, which is a common step in camera calibration. Nevertheless, the calibration pattern used in the work is a cheap print product to achieve flexible calibration, and the accuracy of print product is not usually enough for camera calibration. Thus, to achieve higher calibration accuracy, a non-linear optimization for the coordinates of control points on calibration pattern is carried out first with bundle adjustment of images observed from different orientations and locations.

The optimization is put into effect by minimizing the following function F:

F = \sum_{i} {‖ p_{i} - {\tilde{p}}_{i} (R, t_{s}, m, u_{0}, v_{0}, D) ‖}^{2},

(15)

where p_i is the image coordinate of $P_{i}$ ; ${\tilde{p}}_{i} (R, t_{s}, m, u_{0}, v_{0}, D)$ is the projection of point P_i according to Eqs. (3) and (4); D is the coordinates of control points. The rotation matrix R is parameterized by a vector of three parameters, which is realized by Rodrigues's formula. Lens distortion is not taken into account in the first optimization, because the distortion of telecentric lenses is small and harms the robustness of the optimization for the control points' coordinates. Minimizing function F is a nonlinear minimization problem, which is solved with the Levenberg-Marquardt algorithm. The initial guess of R, t_s, m can be obtained from Section 3.2. The optimization result of D is regarded as the real value of the coordinates of control points in the second non-linear optimization.

The second non-linear optimization is performed to refine all camera parameters considering lens distortion. The optimization function is as follows:

G = \sum_{i} {‖ p_{i} - {\tilde{p}}_{i} (R, t_{s}, m, u_{0}, v_{0}, k) ‖}^{2},

(16)

where $k = {[\begin{matrix} k_{1} & k_{2} & k_{3} & k_{4} & k_{5} \end{matrix}]}^{T}$ is the vector of lens distortion coefficients. The initial guess of R, t_s, m is the same with the first optimization, and the rest initial guess of distortion parameters can be simply set to 0. The flow diagram of algorithm in non-linear optimization is given as follows:

Table 1.

The introduction of double non-linear optimization algorithm

Algorithm: double non-linear optimization

input: R; t _s; m; u₀, v₀; dist; the coordinates of imaging point of control points pts2d; the ideal coordinates of control points pts3d; optimization threshold value τ; the maximum iteration n; optimization step value opt.step. output: The optimized camera parameters opt R, opt. t _s, opt.m, opt.u₀,opt.v₀, opt.dist. 1 p_i ← pts2d; 2 N₁, N₂ ←0; 3 p͂_i ← GetProjection (R, t _s, m, u₀, v₀, dist, pts3d); 4 F ← ∑_i ‖p_i - p͂_i‖²; 5 opt.param1 ← R, t _s, m, u₀, v₀, pts3d; 6 foreach F(opt.param1 ± opt.step) < F(opt.param1) do 7 opt.param1 = opt.param1 ± opt.step; 8 if F < τ or N₁ > n then 9 opt.pts3d ← opt.param1; 10 break 11 else 12 N₁ = N₁ + 1; 13 end 14 end 15 p̑_i ← GetProjection (R, t _s, m, u₀, v₀, dist, opt.pts3d); 16 G ← ∑_i ‖p_i - p̑_i‖²; 17 opt.param2 ← R, t _s, m, u₀, v₀, dist; 18 foreach G(opt.param2 ± opt.step) < G(opt.param2) do 19 opt.param2 = opt.param2 ± opt.step; 20 if G < τ or N₂ > n then 21 opt.R, opt. t _s, opt.m, opt.u₀, opt.v₀, opt.dist ← opt.param2; 22 break 23 else 24 N₂ = N₂ + 1; 25 end 26 end

4 Experiment and Discussion

Computer simulations and real-world experiments have been used to verify the proposed method. The details of simulations and experiments are shown in this section.

4.1 Computer simulations

In the computer simulations, the camera and telecentric lens parameters are set as the same as the standard parameters in real-world experiments, see more details in Section 4.2. The calibration pattern is made by 88 control points (11 × 8). The capture distance is the working distance of the telecentric lenses. Twelve pictures are taken in the simulation with the Euler angles changing randomly.

Gaussian white noise with a mean of 0 and standard deviation σ are added to the control points. The value of σ changes from 0.1 to 1.0 pixels. For each value of σ in the simulation, the calibration procedure is run 100 times and the statistical results are shown in Table 1 and Figure 3.

As shown in the Table 2, the intrinsic parameters set and the calibration results of intrinsic parameters are listed, and the calibration results fit the simulation set well. The error variation of intrinsic parameters is not obvious with the noise strength rise. It is suggested that the method proposed is effective in calibrating intrinsic parameters.

In the figure, the lines show the error variation of extrinsic parameters with the increase of σ, and the means of reprojection errors are also presented. As the results suggest, there is a rising trend for errors of extrinsic parameters and a reprojection error with the increase of σ. However, the errors of extrinsic parameters are low in all simulations, and the means of reprojection errors in both two directions are refined within the noise strength. In addition, the error of tilt is larger than the other two Euler angles in Figure 3(a). The explanation is that tilt expresses the roll here, and the estimation accuracies for the yaw and pitch have a higher precision compared with that of the roll for photogrammetry [28].

Table 2.
The comparison of the intrinsic parameters in simulation set and calibration results

σ M (x10⁻¹) u₀ (pixels) v₀ (pixels) k1 (x10⁻⁴) k2 (x10⁻⁶) k3 (x10⁻⁴) k4 k5 (x10⁻⁴) k5 (x10⁻⁸)

0.1 1.40000 1228.5 1029.5 1.000 −1.000 1.000 −1.000 1.000

0.1 1.40010 1230.6 1031.8 1.003 −1.002 1.009 −0.997 0.998

0.2 1.40022 1233.8 1034.4 0.985 −0.975 0.967 −1.030 0.982

0.3 1.40029 1235.0 1038.2 1.022 −1.064 0.980 −1.033 1.049

0.4 1.40048 1235.7 1038.8 1.017 −1.032 0.984 −1.012 1.021

0.5 1.40051 1238.4 1040.2 1.087 −1.256 0.999 −1.012 1.204

0.6 1.40072 1240.4 1043.8 1.065 −1.111 0.941 −1.066 1.042

0.7 1.40069 1240.5 1044.9 1.109 −1.253 1.046 −0.966 1.158

0.8 1.40089 1243.2 1045.1 1.116 −1.217 0.922 −1.054 1.106

0.9 1.40115 1244.6 1049.6 1.005 −1.073 0.920 −1.052 1.084

1.0 1.40101 1242.6 1049.4 1.039 −1.085 0.946 −1.062 1.031

σ	M (x10⁻¹)	u₀ (pixels)	v₀ (pixels)	k1 (x10⁻⁴)	k2 (x10⁻⁶)	k3 (x10⁻⁴)	k4 k5 (x10⁻⁴)	k5 (x10⁻⁸)
0.1	1.40000	1228.5	1029.5	1.000	−1.000	1.000	−1.000	1.000
0.1	1.40010	1230.6	1031.8	1.003	−1.002	1.009	−0.997	0.998
0.2	1.40022	1233.8	1034.4	0.985	−0.975	0.967	−1.030	0.982
0.3	1.40029	1235.0	1038.2	1.022	−1.064	0.980	−1.033	1.049
0.4	1.40048	1235.7	1038.8	1.017	−1.032	0.984	−1.012	1.021
0.5	1.40051	1238.4	1040.2	1.087	−1.256	0.999	−1.012	1.204
0.6	1.40072	1240.4	1043.8	1.065	−1.111	0.941	−1.066	1.042
0.7	1.40069	1240.5	1044.9	1.109	−1.253	1.046	−0.966	1.158
0.8	1.40089	1243.2	1045.1	1.116	−1.217	0.922	−1.054	1.106
0.9	1.40115	1244.6	1049.6	1.005	−1.073	0.920	−1.052	1.084
1.0	1.40101	1242.6	1049.4	1.039	−1.085	0.946	−1.062	1.031

A comparison simulation is carried out between the proposed method and Chen's method [20]. The main difference between the two is the description of attitude in a non-linear refinement. The proposed method uses the independent variable of attitude description, while the four elements of truncated extrinsic attitude matrices of the camera are adopted in Chen's method. Errors in calibration results are listed in Table 3 when σ is 0.1, 0.5 and 1.0. The data demonstrate that the proposed method performs better in all calibration results.

Figure 3.

The calibration results of computer simulation: (a) the errors of Euler angles; (b) the errors of translation vector; (c) the reprojection error

Table 3.

The errors of calibration results between Chen's method and the proposed method in comparison simulation

σ	Method	M (x10⁻³)	yaw (x10⁻²rad)	pitch (x10⁻²rad)	tilt (x10⁻²rad)	t1 (mm)	t₂ (mm)
0.1	Proposed	0.010	0.006	0.006	0.009	0.052	0.057
0.1	Chen's	0.625	0.221	1.452	3.306	0.155	0.084
0.5	Proposed	0.051	0.030	0.033	0.049	0.244	0.264
0.5	Chen's	1.153	0.395	2.374	5.789	0.732	0.433
1.0	Proposed	0.101	0.061	0.063	0.094	0.347	0.490
1.0	Chen's	1.565	0.499	2.643	6.800	1.377	0.809

4.2 Real-world experiments

The camera and telecentric lens used in the experiment were IGV-B2520M (resolution: 2456×2058; sensor size: 2/3”; cell size: 3.45 μm) and XF-5MDT0.14X178 (magnification: 0.14; distortion: <0.05%), respectively. The calibration pattern used in the experiment is a planar pattern with an additional stage. The planar pattern is a planar board with CALTag [29], and the stage is a circle marker, as shown in Figure 4. The self-identifying pattern is able to deal with the partial visibility due to occlusion or obscuring, and the stage is used to avoid interpretations of parallel projection.

The calibration pattern is set at the range of depth of field and is observed by the camera. In the experiment, 12 pictures are taken to make the calibration, including some pictures taken at the circumstance of a high inclination angle. The planar board is handled with CALTag, while the circle marker is detected by using the ellipse fitting algorithm. The detected result of one picture is shown in Figure 5.

Figure 4.

The calibration images (12 frames)

Figure 5.

One of the captured calibration patterns in the real-world experiment: (a) corner detection results of the calibration pattern; (b) the additional stage detection result of calibration pattern

Figure 6.

The comparison of optimized control points with ideal plane

According to the detected control points, the initial guess of the camera parameters can be achieved by the theory in Sections 3.1 and 3.2. In this calibration approach, a printed paper with control points attached to a cardboard rather than high precision calibration target is adopted, and the errors caused by the printer and the out-of-flatness of cardboard are inevitable. To avoid the mentioned errors, the coordinates of the control points are approved by camera parameters, apart from the lens distortions. The optimized results of the control points' coordinates are shown in Figure 6. In Figure 6, the points represent the control points, and the coloured plane is the ideal plane of planar board, which is the plane of Z = 0. The maximum deviation of the control points is 0.033 mm, and the absolute mean value of deviation is 0.007 mm.

After the optimization of control points and distortion-free camera parameters, the non-linear optimization of all camera parameters then follows. In the second optimization, the refined result of control points is regarded as the real value. The calibration results of the intrinsic parameters are presented in Table 4. With the calibration results, the reprojection errors (δ_u, δ_v) of control points detected are shown in Figure 7, and the arithmetic means of (δ_u, δ_v) are (0.073, 0.073) pixels.

Table 4.

The calibration results of intrinsic parameters

Magnification m	Distortion center (u₀, v₀)	Distortion (k₁, k₂, k₃, k₄, k₅)
0.1391	(920.2, 1273.1)	(−1.713e-6, 8.455e-7, −3.175e-6, 1.237e-6, 1.732e-7)

Figure 7.

The distribution of reprojection errors

The truth of camera parameters cannot be found in the real-world experiments, so the reprojection errors tend to be the main judgments of the experiment's results. Besides, to verify the accuracy of the calibration results, a pair of images is chosen randomly to present the epipolar geometry relationship with the calibration results. Three control points and their corresponding epipolar lines are shown in Figure 8, and the distances between epipolar lines and the corresponding three points are 0.0069 pixels, 0.0003 pixels and 0.0029 pixels, respectively. This result just only depends on the calibration parameters, while the coordinates of control points have no impact on the result. In addition, the coordinates of the control points are measured by the intersection method, and the residuals of every point in the three directions are presented in Figure 9. The results show that the residual of the reconstruction results and the optimized real value stay at a low level for all control points. Combined with the reprojection error, it is suggested that the proposed flexible calibration method is effective and reliable.

Figure 8.

The epipolar geometry relationship: (a) a random calibration frame and three selected control points; (b) the detected locations of control points and their associated epipolar lines on another frame

Figure 9.

The residuals between reconstruction results and the optimized real value for every control point

To explain the performance of the proposed calibration approach, a professional calibration pattern is employed as the comparison. The accuracy of the pattern is within 1 μm, and the comparison of two types of calibration patterns is presented in Figure 10.

Figure 10.

The objective pictures of two types of calibration pattern. Left: the cheap print product; right: the professional customized calibration pattern.

As shown in Figures 11 and 12, pictures are taken to make the calibration a comparison experiment. The data processing is the same as the proposed method except for the double non-linear optimization, because the calibration pattern is accurate enough.

Figure 11.

The calibration images (12 frames) with professional calibration pattern

The calibration results of intrinsic parameters are presented in Table 5. With the calibration results, the reprojection errors (δ_u, δ_v) of control points detected are shown in Figure 12, and the arithmetic means of (δ_u, δ_v) are (0.070, 0.063) pixels.

Table 5.

The calibration results of intrinsic parameters with professional calibration pattern

Magnification m	Distortion center (u₀, v₀)	Distortion (k₁, k₂, k₄, k₅)
0.1396	(1274.9, 924.7)	(3.863e-7, 1.220e-5, −3.109e-5, 1.389e-5, −8.184e-7)

Figure 12.

The distribution of reprojection errors with professional calibration pattern

According to the results of the two comparison experiments, the magnification m and the reprojection errors (δ_u, δ_v) are at the same level. However, there are big differences between the results of the distortion centre and distortion parameters. The main reason for this kind of status is that the distortion for a telecentric lens is quite small. A few small differences in optimization can lead to a big difference in the values of the distortion centre and distortion parameters. Actually, the influence of absolute deviation on the calibration result is not that much. Overall, the comparison experiments verify the performance of the proposed method. The proposed method is able to achieve the accuracy of the professional calibration pattern, but is much more practical and convenient.

5 Conclusions

A flexible two-step calibration method for cameras with double-sided telecentric lenses is proposed in this paper. Firstly, camera parameters with a distortion-free telecentric camera model are achieved by a closed-form solution, in which an additive stage is employed in the calibration pattern to avoid the ambiguity of orthographic projection. Then a non-linear optimization is performed to refine the coordinates of control points with distortion-free camera parameters, and follows another non-linear optimization for all the camera parameters including the distortion coefficients and distortion centre. During the designed non-linear optimization, the camera attitude is parameterized by Rodrigues's formula. Simulation and experimental results illustrate that the proposed method is effective and reliable.

According to the experimental results, it is indicated that the proposed method is able to achieve high-accuracy calibration for a telecentric lens based on a cheap print calibration pattern, which is much more practical and convenient. In addition, similar to Chen's method, the proposed method achieves the calibration of all camera parameters, including intrinsic and extrinsic ones. However, the independent variables of attitude description by Rodrigues's formula, but not the four elements of truncated extrinsic attitude matrices in Chen's method, have been adopted in the nonlinear refinement process. Further, the centre of distortion (u₀, v₀) has been taken into account in the calibration for high accuracy, since the centre of distortion may be significantly displaced from the centre of the image as mentioned previously.

Footnotes

Acknowledgements

The research was supported by the National Natural Science Foundation of China (No.51509251) and Open Fund of Joint Laboratory for Aircraft Measurement and Control (No.FOM2015OF012).

References

Luster

Batchelor

. Telecentric, Fresnel and Micro Lenses. In: Editor, Batchelor

Springer

London, 2012. 259–281.

Watanabe

Nayar

. Telecentric optics for computational vision. Cambridge, United Kingdom: 439. 1996.

Kim

Shin

Chang

Kim

. Development of Image-space Telecentric Lens for Intra-Oral 3D Scanner. J OPT SOC KOREA 2015, 19: 303–310.

Mikš

Novák

. Design of a double-sided telecentric zoom lens. APPL OPTICS 2012, 51: 5928–5935.

Kim

Kanade

. Multiaperture telecentric lens for 3D reconstruction. OPT LETT 2011, 36: 1050–1052.

Ota

Leopold

Noda

Maeno

. Improvement in spatial resolution of background-oriented schlieren technique by introducing a telecentric optical system and its application to supersonic flow. EXP FLUIDS 2015, 56: 48.

Marani

Roselli

Nitti

Cicirelli

D'Orazio

Stella

A 3D vision system for high resolution surface reconstruction. Wellington, New Zealand: IEEE Computer Society, 157–162. 2013.

Baldwin

. Telecentric lens for precision machine vision. Guanajuato, Mex: Society of Photo-Optical Instrumentation Engineers, 440–443. 1996.

Ricolfe-Viala

Sánchez-Salmerón

. Robust metric calibration of non-linear camera lens distortion. PATTERN RECOGN 2010, 43: 1688–1699.

10.

Liu

Tan

Yang

Jia

. Novel approach for laboratory calibration of star tracker. OPT ENG 2010, 49: 73601.

11.

Tsai

. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. Robotics and Automation. IEEE Journal of 1987, 3: 323–344.

12.

Weng

Cohen

Herniou

. Camera calibration with distortion models and accuracy evaluation. IEEE T PATTERN ANAL 1992, 14: 965–980.

13.

Zhang

. A flexible new technique for camera calibration. Pattern Analysis and Machine Intelligence. IEEE Transactions on 2000, 22: 1330–1334.

14.

Fang

Chou

Ding

An Optimal Calibration Method for a MEMS Inertial Measurement Unit. INT J ADV ROBOT SYST 2014, 14.

15.

Schmidt

Kasiński

Kraft

Fularz

Domagała

Calibration of the Multi-camera Registration System for Visual Navigation Bench-marking. INT J ADV ROBOT SYST 2014, 83.

16.

Wei

Shao

Zhang

Wang

. Parallel-based calibration method for line-structured light vision sensor. OPT ENG 2014, 53: 33101.

17.

Liu

Wang

Tan

Yang

Jia

. Autonomous on-orbit calibration of a star tracker camera. OPT ENG 2011, 50: 23604–23608.

18.

Zhu

Liu

Shi

. Accurate 3D measurement system and calibration for speckle projection method. OPT LASER ENG 2010, 48: 1132–1139.

19.

Tian

. An accurate calibration method for a camera with telecentric lenses. OPT LASER ENG 2013, 51: 538–541.

20.

Chen

Liao

Zhang

. Telecentric stereo micro-vision system: Calibration method and experiments. OPT LASER ENG 2014, 57: 82–92.

21.

Hong

Ren

Liu

. Non-iterative method for camera calibration. OPT EXPRESS 2015, 23: 23992–24003.

22.

Hartley

Kang

. Parameter-free radial distortion correction with centre of distortion estimation. Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on. Beijing: IEEE, 1834–1841. 2005.

23.

Miks

Novak

. Dependence of camera lens induced radial distortion and circle of confusion on object position. Optics & Laser Technology 2012, 44: 1043–1049.

24.

Miks

Novak

. Estimation of accuracy of optical measuring systems with respect to object distance. OPT EXPRESS 2011, 19: 14300–14314.

25.

Duane

. Close-range camera calibration. Photo-gram. Eng. Remote Sens 1971, 37: 855–866.

26.

Furferi

Governi

Volpe

Carfagni

. Design and Assessment of a Machine Vision System for Automatic Vehicle Wheel Alignment. INT J ADV ROBOT SYST 2013, 10: 119–134.

27.

Hartley

R ZA

. Multiple View Geometry in Computer Vision. Cambridge, UK: Cambridge University Press, 2004.

28.

Liu

Sun

Zhang

Liu

Zhang

. Hull deformation measurement for spacecraft TT&C ship by Photogrammetry. Science China Technological Sciences 2015, 58: 1339–1347.

29.

Atcheson

Heide

Heidrich

CALTag: High Precision Fiducial Markers for Camera Calibration. Vision, Modeling, and Visualization. Goslar, Germany: The Eurographics Association, 2010.