Paracatadioptric camera calibration based on properties of polar line of infinity point with respect to circle and line

Abstract

Two linear calibration methods based on space-line projection properties for a paracatadioptric camera are presented. Considering the central catadioptric system, a straight line is projected into a circle on the viewing spherical surface for the first projection. The tangent lines in a group at antipode point pairs with respect to the circle are parallel, with the infinity point being the intersection point; therefore, the infinity line can be obtained from two groups of antipode point pairs. Further, the direction of the polar line of an infinity point with respect to the circle is orthogonal to the direction of its infinity point. Hence, on the imaging plane, images of the circular points or orthogonal vanishing points are used to determine the intrinsic parameters. On the basis of the properties of the antipodal point pairs and a least-squares fitting, a corresponding optimization algorithm for line image fitting is proposed. Experimental results demonstrate the robustness of the two calibration methods, that is, for images of the circular points and orthogonal vanishing points.

Keywords

Computer vision 3-D reconstruction and modeling paracatadioptric camera antipodal point pairs orthogonal vanishing points tangent line image of circular points camera intrinsic parameters

Introduction

The rapid development of computer vision technology has caused an increase in the stringency of visual performance requirements.¹ In general, increasing the field of view improves the visual performance of a camera.^2,3 An effective means of enhancing the field of view of a camera is to incorporate mirrors into the camera, as proposed by Hecht and Zajac.⁴ The resulting configuration system is called a catadioptric system. Such a system can be classified into two types on the basis of the presence or absence of a unique effective viewpoint, that is, a central or noncentral system, respectively.⁵ The surface of the mirror of a central catadioptric system is categorized into four types: parabolic, hyperbolic, ellipsoidal, and planar mirrors. The basic feature points in an image captured by a central catadioptric camera can easily be converted into three-dimensional (3-D) coordinates by a projection inverse—a technique broadly applied in vision systems.^4
–6

Geyer and Daniilidis⁶ proposed a unified model of projection image formation for a central catadioptric system that is equivalent to two-step projection by a unit sphere. The calibration methods for central catadioptric cameras can be classified into four categories: self-,^7,8 point-,^9
–11 line-,^{12

–19} and sphere-based calibration.^20

–23 This article considers methods that exploit the projection properties of one line in the unit sphere model to calibrate a paracatadioptric (i.e. with a parabolic mirror) camera.

Many calibration methods that employ 3-D space lines exist for central catadioptric cameras. The advantage of using lines for calibration is that the target scene usually contains lines; thus, a specific template is not required. Geyer and Daniilidis¹² calibrated a paracatadioptric camera using an image of two groups of parallel lines and an image of three lines. They also proposed unifying projection model theory for central catadioptric systems with a unique effective viewpoint.¹³ In their model, the projective lines and points are dual to each other. They also posited a geometric argument in which the center catadioptric system except the planar projection can be calibrated by one image of lines. Further, Barreto and Araujo¹⁴ studied the line images and presented several projective invariant properties of the central catadioptric system using one image of three or more lines to calibrate the camera. The properties include the fact that the intersections of the images of the lines are the principal point, and the polar lines of the principal point with respect to the line images intersect the line image at the image of the circular points. According to the cross ratio invariants of the line image, the mirror parameter is determined. In addition, Wu et al.¹⁵ derived the linear constraints of the line image according to the relationship between the projection of the antipodal points, the projection properties of the lines in the central catadioptric camera, and the cross ratio invariants. Any central catadioptric camera can be calibrated by linear constraints using one image of three or more lines. Ying and Hu¹⁶ proposed a calibration method that uses the geometric invariants of an image of the line and sphere in central catadioptric cameras. One-line image provides three invariants and a sphere provides two invariants. The images of two lines or three spheres can calibrate central catadioptric cameras. Further, Ying and Zha¹⁷ reported that all line images in a central catadioptric system with the same intrinsic parameters belong to the family of conics with two degrees of freedom. They detected line images via the Hough transform and calibrated the camera according to the relationship between the camera intrinsic parameters and the family of conics. Vandeportaele et al.¹⁸ used one image of three or more lines to calibrate paracatadioptric cameras by applying the geometric distance obtained from the intrinsic parameters instead of the algebraic distance. The approach demonstrates that a line is projected to a straight line or circular arc. Finally, Wu et al.¹⁹ proposed a calibration method for paracatadioptric cameras based on lines that are not a conic fitting under a single view. The method establishes sets of linear equations for the camera intrinsic parameters, focal lengths, and skew factor, and the principal point is the image center of the mirror contour.

On the basis of the abovementioned literature analysis, this article presents methods of paracatadioptric camera calibration utilizing the projection of one line and the properties of the polar line of an infinity point with respect to a circle. Considering the unit sphere model, a straight line is projected to a circle of the spherical surface. For a point taken on the circle, the two tangents passing through an antipode point pair are parallel and intersect at the infinity point. Further, the direction of the polar line of an infinity point with respect to the circle is orthogonal to the direction of the infinity point. Therefore, on the image plane, images of the circular points and orthogonal vanishing points can be obtained to determine the camera intrinsic parameters. Then, on the basis of the properties of the antipodal point pairs and a least-squares fitting, an optimization algorithm for line image fitting can be implemented. In this work, two such calibration algorithms are proposed.

The remainder of this article is organized as follows. A review of the unit sphere model for central catadioptric cameras and several related concepts are presented in the next section. Then, two calibration algorithms for paracatadioptric cameras from one line image are described in detail, and an algorithm for line image fitting is presented. The two calibration algorithms are next verified by simulation tests and real experiments. Finally, some concluding remarks are presented.

Preliminaries

In this section, we briefly review the projection process of a central catadioptric system with the unity sphere model,⁶ the relationship of the antipodal image points,¹⁵ and the properties of the polar line of the infinity point with respect to a circle.²⁴

Central catadioptric projection system

As shown in Figure 1, the projection of a 3-D scene point in the central catadioptric camera is equivalent to two-step projection by a unit sphere.⁶ First, in the world coordinate system $O - x_{w} y_{w} z_{w}$ , the 3-D space point $A = {[\begin{matrix} \begin{matrix} x & y \end{matrix} & \begin{matrix} z & 1 \end{matrix} \end{matrix}]}^{T}$ is projected to two different points $A_{\pm} = {[\begin{matrix} \pm x / | | A | | & \pm y / | | A | | & \pm z / | | A | | & 1 \end{matrix}]}^{T}$ with $| | A | | = \sqrt{x^{2} + y^{2} + z^{2}}$ on the unit sphere of a reflective mirror centered at the focal point O ; the two points $A_{\pm}$ constitute a group of antipode points or antipode point pairs. Second, the points $A_{\pm}$ are projected to the two points $a_{\pm}$ on the catadioptric image plane Π by a pinhole camera with the optical center $O_{c}$ . The points $a_{\pm}$ are called a group of antipodal image points. The image plane Π is perpendicular to the line $O O_{c}$ , intersecting at the principal point, and the line $O O_{c}$ is the optical axis of the pinhole camera. The distance between points O and $O_{c}$ is called the mirror parameter distance. The relationship between the mirror type and the value of ξ is as follows: a plane mirror if ξ = 0, an ellipsoid or hyperboloid mirror if 0 < ξ < 1, and a paraboloid mirror if ξ = 1.

Figure 1.

Projection model of the 3-D point A and line L in the central catadioptric system. 3-D: three-dimensional.

Let the pinhole-camera intrinsic parameter matrix be

K = [\begin{matrix} f_{u} & s & u_{0} \\ 0 & f_{v} & v_{0} \\ 0 & 0 & 1 \end{matrix}]

where f_u and f_v are the focal lengths for the u- and v-axis directions, respectively, $[\begin{matrix} u_{0} & v_{0} \end{matrix}]$ are the coordinates of the principal point in the image plane, and s is the skew factor for the u and v axes. The antipodal points $A_{\pm}$ are projected to the catadioptric image points $a_{\pm}$ as follows

λ_{1, 2} a_{\pm} = K [\begin{matrix} I & t \end{matrix}] A_{\pm} = K ({\tilde{A}}_{\pm} + {[\begin{matrix} 0 & 0 & ξ \end{matrix}]}^{T})

where λ ₁ and λ ₂ are two nonzero scale factors, t is the translation vector between the world and camera coordinate system, and ${\tilde{A}}_{\pm} =$ ${[\begin{matrix} \pm x / ‖ A ‖ & \pm y / ‖ A ‖ & \pm z / ‖ A ‖ \end{matrix}]}^{T}$ denotes the nonhomogeneous coordinates of $A_{\pm}$ .

Similarly, the projection of a 3-D scene line in the central catadioptric camera is equivalent to two-step projection by a unit sphere model. First, a 3-D space line L is projected to a circle S on the unit sphere through the viewpoint O ; second, the circle S is projected to the conic C on the image plane through the virtual camera O _c. This projection process is shown in Figure 1.

Proposition 1

In Figure 1, if ${a_{+}, a_{-}}$ is a group of antipodal image points in a central catadioptric system, their relationship is as follows¹⁵

\frac{1 + \sqrt{1 + τ a_{+}^{T} ω a_{+}}}{a_{+}^{T} ω a_{+}} a_{+} + \frac{1 + \sqrt{1 + τ a_{-}^{T} ω a_{-}}}{a_{-}^{T} ω a_{-}} a_{-} = 2 p

where $τ = (1 - ξ^{2}) / ξ$ and $ω = K^{- T} K^{- 1}$ is the image of the absolute conic (IAC). In the paracatadioptric system, equation (3) can be simplified to

\frac{1}{a_{+}^{T} ω a_{+}} a_{+} + \frac{1}{a_{-}^{T} ω a_{-}} a_{-} = p

Properties of the polar line of infinity point with respect to a circle

Definition

Given a point x and conic C , define a line $l = C x$ . The line l is called the polar of x with respect to C , and the point x is the pole of l with respect to C .²⁴

Proposition 2

The direction of the polar line of an infinity point with respect to a circle and the direction of the infinity point are orthogonal.

Proof

As shown in Figure 2, the polar line of the infinity point V with respect to the circle O is the diameter r along which the direction has an infinity point V ′. The lines $r_{1}$ and $r_{2}$ that pass through the two endpoints A and B of r are tangential to O ; therefore, $r_{1} ∥ r_{2}$ and intersect at the infinity point V .²⁴ The tangents $r_{1}$ and r are orthogonal; thus, V and $V^{'}$ form a group of orthogonal infinity points depending on the properties of the projective geometry. □

Figure 2.

Properties of the polar line of infinity point with respect to a circle.

Paracatadioptric camera calibration and line image fitting

In this section, through an analysis of the projection properties of one line for a paracatadioptric camera, the manner in which the properties of the polar line of the infinity point with respect to a circle can be used for the calibration of a paracatadioptric camera is explained, which corresponds to proposition 3 (for orthogonal vanishing points) and proposition 4 (for imaged circular points) in the following. According to the properties of the antipodal points and a least-squares fitting, an optimization algorithm for line image fitting is proposed.

Properties of the line image for a paracatadioptric camera

Proposition 3

For a central catadioptric system, given two different points on the line image, we can obtain two groups of orthogonal vanishing points.

Proof

As shown in Figure 1, for the unit sphere model, the projection of the line L is a circle S on the unit spherical surface, with the center of the circle corresponding to the unity sphere center O . We take the antipodal points $A_{\pm}$ on the projection circle S , which are the two endpoints of a diameter of S , in accordance with the central catadioptric projection model. The tangents of S at $A_{\pm}$ are parallel and intersect at the infinity point $D_{1 \infty}$ according to the properties of the diameter of a circle. Similarly, we take a pair of antipodal points $B_{\pm}$ on S . The tangents of S at points $B_{\pm}$ are parallel and intersect at another infinity point $D_{2 \infty}$ . The line passing through D _1∞ and D _2∞ is the infinity line $L_{\infty}$ . The polar lines of D _1∞ and D _2∞ with respect to S and the infinity line $L_{\infty}$ intersect at the infinity points $D'_{1 \infty}$ and ${D^{'}}_{2 \infty}$ . According to proposition 2, the infinity points D _i _∞ and $D'_{i \infty}$ (i = 1, 2) form a group of orthogonal infinity points.

As shown in Figure 3, on the catadioptric image plane, the line image is denoted by C , and the images of the points A ₊ and A _– are $a_{+}$ and $a_{-}$ , respectively. The tangents of C at $a_{+}$ and $a_{-}$ intersect at the vanishing point $d_{1}$ , which is the image of the infinity point $D_{1 \infty}$ , with

d_{1} = (C \cdot a_{+}) \times (C \cdot a_{-})

where × is the vector product and ⋅ is the product. We denote the images of B ₊ and B _– by $b_{+}$ and $b_{-}$ , respectively. The tangents of C at $b_{+}$ and $b_{-}$ intersect at the vanishing point $d_{2}$ , which is the image of the infinity point $D_{2 \infty}$ , with

d_{2} = (C \cdot b_{+}) \times (C \cdot b_{-})

Figure 3.

Polar line of a vanishing point with respect to a line image.

The vanishing line l passes through $d_{1}$ and $d_{2}$ on the plane containing S , being the image of the infinity $L_{\infty}$

l = d_{1} \times d_{2}

The polar lines of $d_{1}$ and $d_{2}$ with respect to C intersect l at the vanishing points ${d^{'}}_{1}$ and ${d^{'}}_{2}$ , respectively, which are the images of the infinity points ${D^{'}}_{1 \infty}$ and ${D^{'}}_{2 \infty}$ , respectively. Thus

{d^{'}}_{1} = (C \cdot d_{1}) \times l

{d^{'}}_{2} = (C \cdot d_{2}) \times l

On the basis of proposition 2, d _i _∞ and $d'_{i \infty}$ (i = 1, 2) constitute a group of orthogonal vanishing points. □

The choice of the locations of the images points $a_{+}$ and $b_{+}$ is arbitrary, but they might be as far away from each other as possible. If the distance between them is too close, the distance between vanishing points $d_{1}$ and $d_{2}$ is too small, and they may even coincide because of the noise effect. Thus, the accuracy of the vanishing line l is decreased.

Proposition 4

For a central catadioptric system, given two different points on the line image, the images of the circular points can be obtained.

Proof

For proposition 3, the vanishing line l can be obtained. Thus, l intersects the conic C at the images of the circular points $m_{I}$ and $m_{J}$ .²⁴

Line image fitting in a central catadioptric system

The image of a straight line is conic in a central catadioptric system, but only a section of the conic is visible in the image plane. Therefore, the line image fitting is inaccurate when only a section of the conic is known.

Proposition 5

In a central catadioptric system (see Figure 1), with knowledge of the image point $a_{i +}$ and its antipodal image point $a_{i -}$ $(i \geq 5)$ , conic fitting can be performed.

Proof

Let the conic C be the image of a straight line observed with a central catadioptric system. We take $N (N \geq 5)$ points $a_{i +}$ on the visible section of the conic with $i = 1, 2, 3, \dots, N$ . The projective contour of the mirror can be completely extracted, and the equation of the projective contour of the mirror can be formulated through a least-squares fitting to initialize the camera intrinsic parameters $\tilde{K}$ ¹⁶; hence, according to proposition 1, the antipodal image point $a_{i -}$ of point $a_{i +}$ can be calculated. Let the conic C be

C = a x^{2} + b x y + c y^{2} + d x + e y + f = 0

We minimize the algebraic distance $C (x_{j}, y_{j})$ from the antipodal point pairs $a_{i \pm}$ to C ²⁵; then

Γ = \sum_{j = 1}^{2 N} C^{2} (x_{j}, y_{j})

where (x_j , y_j ) are the coordinates of a _i+ or a _i−. We minimize the objective function in equation (11), to which there always exists the trivial solution a = b = c = d = e = d = 0. In order to avoid the zero solution, we normalize $C (x, y)$ with a = 1, b = 1, c = 1, d = 1, e = 1, and f = 1. When a = 1, let $X_{1} = {[b c d e f]}^{T}$ ; thus, the equation $C (x_{j}, y_{j}) = 0$ becomes $a_{j}^{T} X_{1} - b_{j} = 0$ , where $a_{j} = {[x_{j} y_{j} y_{j}^{2} x_{j} y_{j} 1]}^{T}$ and $b_{j} = - x_{j}^{2}$ . Given 2N points, we obtain the following equation

A X_{1} = b

where $A = {[a_{1} a_{2} \dots a_{2 N}]}^{T}$ and $b = {[b_{1} b_{2} \dots b_{2 N}]}^{T}$ . By minimizing the objective function in equation (11), we obtain

Γ (X_{1}) = {(A X_{1} - b)}^{T} (A X_{1} - b)

We then minimize the objective function; that is, we obtain the partial derivative of equation (13) with respect to $X_{1}$ , where the partial derivative is zero

2 A^{T} (A X_{1} - b) = 0

The solution of equation (14) is

X_{1} = {(A^{T} A)}^{- 1} A^{T} b

and the coefficients of the conic are $X_{1}^{1} = {[\begin{matrix} 1 & X_{1}^{T} \end{matrix}]}^{T}$ .

Similarly, through normalization with b = 1, c = 1, d = 1, e = 1, and f = 1, the coefficients of a conic, that is, $X_{2}^{1}, X_{3}^{1}, X_{4}^{1}, X_{5}^{1}, and X_{6}^{1}$ , can be obtained; thus, the equation of the line image can be determined from

X = \frac{1}{6} \sum_{i = 1}^{6} X_{i}^{1}

Line-based calibration method for a paracatadioptric camera

A group of orthogonal vanishing points d ₁ and ${d^{'}}_{1}$ , whose homogeneous coordinates are $d_{1} = {[\begin{matrix} u_{1} & v_{1} & 1 \end{matrix}]}^{T}$ and ${d^{'}}_{1} = {[\begin{matrix} {u^{'}}_{1} & {v^{'}}_{1} & 1 \end{matrix}]}^{T}$ , can provide one linear constraint on the IAC.²⁴ The constraint is as follows

d_{1}^{T} ω {d^{'}}_{1} = 0

A group of circular points $I {[\begin{matrix} 1 & i & 0 \end{matrix}]}^{T}$ and $J {[\begin{matrix} 1 & - i & 0 \end{matrix}]}^{T}$ constitutes two special points on the absolute conic, and their images $m_{I} = {[\begin{matrix} x_{1} + x_{2} i & y_{1} + y_{2} i & 1 \end{matrix}]}^{T}$ and $m_{J} = {[\begin{matrix} x_{1} - x_{2} i & y_{1} - y_{2} i & 1 \end{matrix}]}^{T}$ lie on the IAC, which yields the following

m_{I}^{T} ω m_{I} = 0, m_{J}^{T} ω m_{J} = 0

The points m _I and m _J are a group of conjugate points; thus, equation (18) only provides two linear constraints on the IAC²⁶

Re (m_{I}^{T} ω m_{I}) = 0, Im (m_{I}^{T} ω m_{I}) = 0

where Re and Im represent the real and imaginary parts, respectively.

Note that a group of orthogonal vanishing points provides one linear constraint on the IAC, but all of the orthogonal vanishing points for a plane only provide two constraints on the IAC.²⁴ A group of the images of the circular points can provide two constraints on the IAC. Therefore, at least three views of one line should be provided to satisfy propositions 3 and 4.

The two algorithms proposed in this study, that is, for the images of the circular points (proposition 4) or orthogonal vanishing points (proposition 3), consist of the following steps:

Step 1: Views including at least one line image are input, and the pixel coordinates of the line image and the projective contour of the mirror are extracted in $n (n \geq 3)$ views; the equation of the projective contour of the mirror can be formulated through a least-squares fitting to initialize the camera’s intrinsic parameters $\tilde{K}$ and the IAC $\tilde{ω} = {\tilde{K}}^{- T} {\tilde{K}}^{- 1}$ ¹⁶; the equations of the line images are estimated using Proposition 5.

Step 2: Two different points, a ₊ and b ₊, are taken on every line image, and their antipodal image points, a _– and b _–, are determined using equation (4); the vanishing points d ₁ and d ₂ are determined using equations (5) and (6).

Step 3: The vanishing line l is determined using equation (7); the orthogonal vanishing points ${d^{'}}_{1}$ and ${d^{'}}_{2}$ are obtained using equations (8) and (9).

Step 4: The two common intersection points (i.e. the imaged circular points) for the vanishing line l and line image C are calculated using the “solve” function in MATLAB.

Step 5: (a) When the orthogonal vanishing points are known, solve for ω using equation (17).

(b) When the images of the circular points are known, solve for ω using equation (19).

Step 6: Determine K by performing the inverse via Cholesky factorization of ω .

Experiments

We tested the validity and robustness of two calibration algorithms via simulation tests and real experiments. Further, we compared the results with Barreto and Araujo’s¹⁴ three-line method and Wu et al.’s¹⁵ calibration methods. In the following, the methods derived from propositions 3 and 4 are referred to as “Ours 1” and “Ours 2,” respectively.

Experiment using simulation data

Let the simulated central catadioptric camera be a paracatadioptric camera, that is, ξ = 1, with the five intrinsic parameters being $f_{u} = 700$ , $f_{v} = 710$ , s = 0.4, $u_{0} = 450$ , and $v_{0} = 450$ . Three images are needed in our calibration methods. Figure 4 shows three line images with the simulated camera; the red thick circle is the projection of the mirror contour, and the three purple curves (thin lines) are the three line images.

Figure 4.

Simulated line images.

In the simulation experiment, a line image was fit according to the method presented in proposition 5 (step 1 of both algorithms). Five hundred points were chosen on each line image, with 200 points chosen on the mirror projective contour. To test the sensitivity of the calibration methods to noise, Gaussian noise with a zero mean and a standard deviation σ of zero to three pixels was added to the chosen pixel coordinates. After the noise was added, we compared the difference in the results given by our proposed methods (corresponding to propositions 3 and 4) and those reported in the literature.^14,15 For each noise level, 1000 independent trials were performed to determine the absolute error of the intrinsic parameters. The average results of the absolute error are shown in Figure 5(a) to (e). The results indicate that, when σ is zero, the absolute errors of the intrinsic parameters are zero; the absolute errors of our two methods and the methods of Barreto and Araujo¹⁴ and Wu et al.¹⁵ increase virtually linearly with the noise level σ. However, the absolute errors of our proposed algorithms increase more slowly than those of the other calibration algorithms. The contrast in the simulation experimental results confirms that our proposed methods are feasible.

Figure 5.

Variations in the absolute errors of the intrinsic parameters with different calibration methods under varying levels of Gaussian noise.

We compared the runtimes of the four methods using MATLAB 2016a implementations of all algorithms on a computer equipped with a Pentium IV processor running at 2.1 GHz and 2.0 GB of RAM. We performed 1000 independent trials to obtain the average values of the runtimes, the results of which are listed in Table 1. The average value of the runtimes of Wu et al.¹⁵ is far greater than the other methods because the generalized inverses of antipodal image points needed to be computed before establishing the linear constraints between the antipodal image points and the IAC.

Table 1.

Runtimes for the four algorithms (unit: seconds).

	Ours 1	Ours 2	Barreto and Araujo¹⁴	Wu et al.¹⁵
Runtime	0.022	0.024	0.025	0.032

Experiment with real data

In the real experiment, a checkerboard was used to calibrate the model to replace the lines. A catadioptric camera shot the scene from different viewing angles, and three pictures were selected as real images. Three experimental images with resolutions of 3680 × 3488 pixels are shown in Figure 6(a) to (c). First, the edges were detected using the Canny algorithm to detect the edges of the images, as shown in Figure 7(a) to (c). The equations of the line images were obtained using the method given in proposition 5 (step 1 of both proposed algorithms). Then, the intrinsic camera parameters were obtained on the basis of our calibration algorithms and the methods of Barreto and Araujo¹⁴ and Wu et al.¹⁵ The calibration results are listed in Table 2 for the real experiments. Note that the calibration results exhibit little difference; thus, our calibration methods are efficient. To further check the accuracy of the calibration results in Table 2, Figure 6(a) is taken as an example and rectified to its projective image using the determined intrinsic parameters, as shown in Figure 8(a) to (d). Again, the results indicate that our proposed calibration algorithms are effective.

Figure 6.

Three checkerboard images captured by a paracatadioptric camera.

Figure 7.

Resulting edges of the three images extracted in Figure 6.

Table 2.

Comparison of the calibration results of the intrinsic parameters with different methods (unit: pixels).

	f_u	f_v	s	u ₀	v ₀
Ours 1	1332.42	1302.76	2.13	1842.82	1744.12
Ours 2	1335.59	1305.86	2.47	1843.56	1742.62
Barreto and Araujo¹⁴	1384.64	1381.49	3.01	1838.92	1740.60
Wu et al.¹⁵	1347.90	1346.85	2.71	1843.25	1747.23

Figure 8.

Image results rectified using the intrinsic parameters listed in Table 2 for (a) Ours 1, (b) Ours 2, (c) Barreto and Araujo,¹⁴ and (d) Wu et al.¹⁵

In order to validate the accuracy of the calibration results in Table 2, the checkerboard pattern in Figure 6 was reconstructed. A valid region of the checkerboard pattern images from Figure 6(a) to (c) was chosen. On the basis of the Harris corner detector,²⁷ the feature points of the checkerboard shown in Figure 9(a) to (c) were selected. Applying the epipolar geometry constraint algorithms for two views of the paracatadioptric camera,²⁸ 52 corresponding points were selected from the two visible sides of the checkerboard image and were then used for reconstruction (see Appendix 1). The reconstruction results obtained on the basis of the data listed in Table 2 are shown in Figure 10(a) to (d). From Figure 10, it is clear that the reconstructed 3-D points are coplanar for the two sides.

Figure 9.

(a) to (c) Results of Harris corner detection for Figure 7(a) to (c), respectively.

Figure 10.

Reconstruction results obtained using the intrinsic parameters in Table 2 for (a) Ours 1, (b) Ours 2, (c) Barreto and Araujo,¹⁴ and (d) Wu et al.¹⁵

The parallelism and orthogonality in Figure 10(a) to (d) were also checked. First, according to least-squares fittings, the equations of the lines for every row or column in Figure 10(a) to (d) were obtained. Then, the average of the angles between any two points in parallel directions was calculated. Similarly, the average of the angle for the orthogonal direction was calculated. The results for the angles obtained from Figure 10 are summarized in Table 3. The real angle for the parallel line direction is 0°, and that for the orthogonal direction is 90° on the calibration checkerboard pattern. The 3-D reconstruction angles are similar to the true values in the parallel or orthogonal direction. This indirectly proves that the calibration results of the intrinsic parameters in Table 2 are effective.

Table 3.

Angles of the parallel and orthogonal directions recovered from Figure 10 (units: degrees).

	Ours 1	Ours 2	Barreto and Araujo¹⁴	Wu et al.¹⁵
Parallel	1.0032	0.9528	1.0120	1.0302
Orthogonal	88.8192	89.0063	88.3103	88.5824

Conclusion

A camera calibration algorithm is often related to a vanishing point or an image of circular points. In this work, for application to three images containing one line, two calibration methods based on the properties of the polar line of an infinity point with respect to a circle in a paracatadioptric camera and incorporating the orthogonal vanishing points or the images of the circular points were proposed. In this approach, for the unit sphere model, a straight line is projected to a circle on the unit sphere. The tangents of the circle at antipodal point pairs are parallel and intersect at an infinity point. Then, an infinity line can be obtained according to two groups of antipodal points. The direction of the polar line of an infinity point with respect to a circle and the direction of the infinity point are orthogonal. Further, the intersection point of the infinity line and the circle is a circular point. According to the projective invariance, the images of the circular points and the orthogonal vanishing points can be determined. The camera intrinsic parameters can then be obtained according to the relationship between the images of the circular points or orthogonal vanishing points and the intrinsic parameters. The experimental results verify the robustness of our proposed calibration methods. An optimization method based on the fitting of one line image according to the relationship between antipodal points was also proposed.

In the literature,¹⁴ on the basis of the geometric invariance under a projective transformation, one image containing three or more lines is used to calibrate the camera. A pair of lines is projected into two circles and intersect at a pair of antipodal points on the viewing spherical surface. Three lines are projected into three circles, which intersect at three pairs of antipodal points, and the intersection point of the three lines passing though the antipodal points is the center of the three circles. The polar line of the center of the circle with respect to the circle is the infinity line, which intersects the circle at the circular points. On the basis of the geometric invariance under a projective transformation, the three lines passing through the intersection points corresponding to the line images intersect at the principle point, that is, the image of the center of the circle. The polar lines of the principal point with respect to the line image intersect the line image at the image of the circular points. The images of the circular points are used to determine the intrinsic parameters. In the literature,¹⁵ a relationship between the projection on the viewing sphere of a space point and its image was derived to obtain a linear constraint between the antipodal image points and the IAC. First, the principal point was determined by the intersecting point of the three line images. Then, two pairs of antipodal image points were taken from each of the three line images according to the constraint, and central catadioptric cameras were calibrated for a single view.

In this study, the elements at infinity are obtained on the basis of both the properties of the tangent lines of a circle and the properties of the polar line of an infinity point with respect to a circle.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (NSFC) (61663048 and 11361074).

ORCID iD

Yue Zhao

Appendix 1

References

Yao

Liu

. A flexible calibration approach for cameras with double-sided telecentric lenses. Int J Adv Robot Syst 2016; 13: 1–9.

Bazin

Vasseur

Kweon

. Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment. Int J Robot Res 2012; 31(1): 63–81.

Zhu

Cao

. A special unique solution case of the perspective-three-point problem for external parameter calibration of an omnidirectional camera. Int J Adv Robot Syst 2012; 9: 1–9.

Hecht

Zajac

. Optics. 3rd ed. Reading, MA: Addison-Wesley Publishing Company, 1997.

Baker

Nayar

. A theory of single-viewpoint catadioptric image formation. Int J Comput Vision 1999; 35(2): 175–196.

Geyer

Daniilidis

. Catadioptric projective geometry. Int J Comput Vision 2001; 45(3): 223–243.

Kang

. Catadioptric self-calibration. In: Proceedings of the 2000 IEEE international conference on computer vision and pattern recognition, Hilton Head Island, USA, 13–15 June 2000, pp. 201–207. IEEE. DOI: 10.1109/CVPR.2000.855820.

Geyer

Daniilidis

. Structure and motion from uncalibrated catadioptric views. In: Proceedings of the 2001 IEEE international conference on computer vision and pattern recognition, Kauai, HI, USA, 8–14 December 2001, pp. 279–286. IEEE. DOI: 10.1109/CVPR.2001.990487.

Scaramuzza

Martinelli

Siegwart

. A flexible technique for accurate omnidirectional camera calibration and structure from motion. In: Proceedings of the 2006 IEEE international conference on computer vision systems, New York, NY, USA, 4–7 January 2006, pp. 45–53. IEEE. DOI: 10.1109/ICVS.2006.3.

10.

Gasparini

Sturm

Barreto

. Plane-based calibration of central catadioptric cameras. In: Proceedings of the 2009 IEEE international conference on computer vision, Kyoto, Japan, 29 September–2 October 2009, pp. 1195–1202. IEEE. DOI: 10.1109/ICCV.2009.5459336.

11.

Deng

. An easy calibration method for central catadioptric cameras. Acta Automatica Sinica 2007; 33(8): 801–808.

12.

Geyer

Daniilidis

. Catadioptric camera calibration. In: Proceedings of the 1999 IEEE international conference on computer vision, Kerkyra, Greece, 20–27 September 1999, pp. 398–404. IEEE. DOI: 10.1109/ICCV.1999.791248.

13.

Geyer

Daniilidis

. A unifying theory for central panoramic systems and practical implications. In: Proceedings of the 2000 European conference on computer Vision, Dublin, Ireland, 26 June–1 July 2000, pp. 445–461. Springer.

14.

Barreto

Araujo

. Geometric properties of central catadioptric line images and their application in calibration. IEEE Trans Pattern Anal Mach Intell 2005; 27(8): 1327–1333.

15.

Duan

. A new linear algorithm for calibrating central catadioptric cameras. Pattern Recognit 2008; 41(10): 3166–3172.

16.

Ying

. Catadioptric camera calibration using geometric invariants. IEEE Trans Pattern Anal Mach Intell 2004; 26(10): 1260–1271. IEEE.

17.

Ying

Zha

. Simultaneously calibrating catadioptric camera and detecting line features using Hough transform. In: Proceedings of the 2005 IEEE international conference on intelligent robots and systems, Edmonton, Alta., Canada, 2–6 August 2005, pp. 412–417. IEEE. DOI: 10.1109/IROS.2005.1545166.

18.

Vandeportaele

Cattoen

Marthon

. A new linear calibration method for paracatadioptric cameras. In: Proceedings of the 2006 IEEE international conference on pattern recognition, Hong Kong, China, 20–24 August 2006, pp. 647–651. IEEE. DOI: 10.1109/ICPR.2006.121.

19.

. Easy calibration for para-catadioptric-like camera. In: Proceedings of the 2006 IEEE international conference on intelligent robots and systems, Beijing, China, 9–15 October 2006, pp. 5719–5724. IEEE. DOI: 10.1109/IROS.2006.282377.

20.

Ying

Zha

. Identical projective geometric properties of central catadioptric line images and sphere images with applications to calibration. Int J Comput Vision 2008; 78(1): 89–105.

21.

Duan

. A calibration method for paracatadioptric camera from sphere images. Pattern Recognit Lett 2012; 33(6): 677–684.

22.

Zhao

Wang

. Intrinsic parameter determination of a paracatadioptric camera by the intersection of two sphere projections. J Opt Soc Am A 2015; 32(11): 2201–2209.

23.

Ying

Zha

. Linear approaches to camera calibration from sphere images or active intrinsic calibration using vanishing points. In: Proceedings of the 2005 IEEE international conference on computer vision, Beijing, China, 17–21 October 2005, pp. 596–603. IEEE. DOI: 10.1109/ICCV.2005.145.

24.

Hartley

Zisserman

. Multiple view geometry in computer vision. 2nd ed. Cambridge: Cambridge University Press, 2003.

25.

Zhang

. Parameter estimation techniques: a tutorial with application to conic fitting. Image Vision Comput 1997; 15(1): 59–76.

26.

Meng

. A new easy camera calibration technique based on circular points. Pattern Recognit 2003; 36(5): 1155–1164.

27.

Zhang

. An improved Harris corner detection algorithm for noised images. In: Proceedings of the 2012 international conference on materials science and information technology, Advanced Materials Research, 3 January 2012, pp. 6151–6156. Trans Tech Publications.

28.

Micusik

Pajdla

Para-catadioptric camera auto-calibration from epipolar geometry. In: Proceedings of the 2004 Asian conference on computer vision, Jeju Island, Korea, 28–30 January 2004, pp. 748–753. Asian Federation of Computer Vision Societies.

29.

Zhang

. Self-recalibration of a structured light system via plane-based homography. Pattern Recognit 2007; 40(4): 1368–1377.

30.

Zhang

. A method for calibrating the central catadioptric camera via homographic matrix. In: Proceedings of the 2008 IEEE international conference on information and automation, Zhangjiajie, China, 20–23 June 2008, pp. 972–977. IEEE. DOI: 10.1109/ICINFA.2008.4608140.