Keywords

1 Introduction

Because of its high accuracy, timeliness, and other advantages, vision sensors measurement is widely used in industrial measurement in recent years [1, 2]. MVS measurement technology has obvious advantages in the industrial measurement. In MVS system, each sensor has independent coordinates, which make it impossible for different sensors to describe the scene in consistency, unless a global calibration is achieved.

Researches on the global calibration method of MVS have been done by lots of scholars and various solutions have been proposed. Kumar etc used plane mirror to achieve the global calibration [3]. This method requires a strict restriction for the location of the mirrors and the precision of calibration is not stable. Zhang Guang-jun etc suggested the global calibration based on 1D target [4]. It is simple, but only one feature point can be obtained in each moving of the target and the extracting accuracy of feature point on target cannot be guaranteed. Hu Hao proposed a precise camera calibration method which based on close-range photogrammetry technology [5]. But this technique needs a large target and not portable, so it is inconvenient to use.

This paper proposed a simple and practical method of MVS’s global calibration. The system just need a checkerboard which can be printed in lab and a structured light, What’s more, this system has a lot of advantages. For example, it can calibrate the relationship of the position of all sensors quickly, the times of coordinate transformation in the proposed method are few. It has a high precision which has been confirmed by experiments.

2 Mathematical Model of Structured Light Vision Sensor

Figure 1 shows the geometrical model [6] of a structured light. The world coordinate frame and the camera coordinate frame o c _x c y c z c are the same, and the image plane coordinate are defined as O-XY. The relation between the world coordinates and the image coordinates can be illustrated by (1).

Fig. 1
figure 1

Geometrical model of a structured light

Full size image
$$ \left\{\begin{array}{l}\rho \left[\begin{array}{c}\hfill X\hfill \\ {}\hfill Y\hfill \\ {}\hfill 1\hfill \end{array}\right]=A\left[\begin{array}{c}\hfill x\hfill \\ {}\hfill y\hfill \\ {}\hfill z\hfill \end{array}\right]\\ {} ax+ by+ cz+d=0\end{array}\right. $$
(1)

\( A=\left[\begin{array}{ccc}\hfill {\alpha}_x\hfill & \hfill 0\hfill & \hfill {\mu}_0\hfill \\ {}\hfill 0\hfill & \hfill {\alpha}_y\hfill & \hfill {\nu}_0\hfill \\ {}\hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right] \) is the known intrinsic matrix of the camer.ρis a unknown scale. ax + by + cz + d = 0 is the function of the light plane in the world coordinate.

3 Global Calibration for Multi-vision Sensors

3.1 Calibration of Single Structured Light Vision Sensor

The process to calibrate a structured light vision sensor includes two parts: camera calibration and projector calibration.

As the camera calibration, we use Zhang’s method [7] to get camera intrinsic parameters and distortion factors [8]. In the experiment, we will correct image distortion before processing image.

As the projector calibration method, we use the method in the literature [6] and the method introduction is as follows.

Firstly we will determine the world coordinates of the target feature points. A plane chessboard is selected as the target. Corner points in each line of the chessboard can be seen as a 1D target, such as A B C D shown in Fig. 1. The laser should intersect with the target. The world coordinates of A C D are denoted by (x A , y A , z A ), (x C, y C, z C) and (x D, y D, z D). According to the relative positional relationship between the three points, we obtain the following equation:

$$ {\left({x}_C,{y}_C,{z}_C\right)}^T={\left({x}_A,{y}_A,{z}_A\right)}^T+h{\left(\left({x}_D,{y}_D,{z}_D\right)-\left({x}_A,{y}_A,{z}_A\right)\right)}^T $$
(2)

Where h is a known parameter. The image coordinates of A, C and D are denoted by (X A , Y A ), (X C, Y C ) and (X D , Y D ), and they can be got from the Chessboard target. Depending on the camera imaging model and (2), we can get (3):

$$ \begin{array}{l}{\alpha}_x{x}_A-\left({X}_A-{\mu}_0\right){z}_A=0\\ {}{\alpha}_y{y}_A-\left({Y}_A-{\nu}_0\right){z}_A=0\\ {}{\alpha}_x{x}_D-\left({X}_D-{\mu}_0\right){z}_D=0\\ {}{\alpha}_y{y}_D-\left({Y}_D-{\nu}_0\right){z}_D=0\\ {}{\alpha}_x\left(1-h\right){x}_A-\left(1-h\right)\left({X}_C-{\mu}_0\right){z}_A+{\alpha}_xh{x}_D-h\left({X}_C-{\mu}_0\right){z}_D=0\\ {}{\alpha}_y\left(1-h\right){y}_A-\left(1-h\right)\left({Y}_C-{\nu}_0\right){z}_A+{\alpha}_yh{y}_D-h\left({Y}_C-{\nu}_0\right){z}_D=0\end{array} $$
(3)

Because the distance L between A and D is known, we can get (4)

$$ \left\Vert \left({x}_D,{y}_D,{z}_D\right)-\left({x}_A,{y}_A,{z}_A\right)\right\Vert =L $$
(4)

According to (3) and (4), we can get the world coordinates of the target feature points A and D. Then the world coordinates of point C can be determined by (2).

Secondly we will calculate the world coordinates of light plane feature point. The advantages of using corner points in each line of the chessboard as a 1D target are listed as follows. Firstly, at each position, more than one light plane feature points can be got. Secondly, the image coordinates of feature points in light plane can be determined by two intersecting lines: one line is fit by the structured light and the other is matched through each row corner points in chessboard, such as corner points A C D shown in Fig. 1. This can improve the accuracy of the image coordinates of light plane feature point. The image coordinates of B is denoted by (X B , Y B ). According to the camera imaging model, we can get (5).

$$ \rho {\left({X}_B,{Y}_B,1\right)}^T=A{\left({x}_B,{y}_B,{z}_B\right)}^T $$
(5)

The relative positional relationship of A, B and D is described in (6)

$$ {\left({x}_B,{y}_B,{z}_B\right)}^T={\left({x}_A,{y}_A,{z}_A\right)}^T+{h}_x{\left(\left({x}_D,{y}_D,{z}_D\right)-\left({x}_A,{y}_A,{z}_A\right)\right)}^T $$
(6)

Where h x is unknown. According to (5) and (6), we can get h x and obtain the world coordinates of point B by (6).

Thirdly we will get light plane coefficients assuming. we freely place the target at n positions within the FOV, light plane feature points denoted as (x Bi , y Bi , z Bi ) (i = 1,2,3… kn) can be obtained and k can be determined by the size of the chessboard. According to (1), the following objective function are created:

$$ e(t)={\displaystyle \sum_{i=1}^{kn}F\left({x}_{Bi},{y}_{Bi},{z}_{Bi}\right)} $$
(7)

Where F(x Bi , y Bi , z Bi ) = (ax Bi  + by Bi  + cz Bi  + d)2. Using the least squares method, light plane coefficients a, b, c and d can be got. Calibration of single structured light vision sensor is finished.

3.2 Global Calibration for MVS

Figure 2 shows the arrangement of MVS system. ViCF (i = 1,2,3) represented the coordinate system corresponding to each camera respectively. GCS was established in V1CF, and transformed V2CF, V3CF to V1CF coordinate system respectively to complete the global calibration of MVS.

Fig. 2
figure 2

The arrangement of MVS system

Full size image

The key to the conversion between two coordinate systems is to obtain the [R, t] matrix that represents a relative positional relationship [9]. Take the solving of [R 21 , t 21 ] between the local coordinate system V2CF and the global coordinate system V1CF as an example, then illustrated the idea of global calibration.

As shown in Fig. 2, the point P on the Light-receiving surface was marked (X P1 , Y P1 ) in the camera 1, and marked (x P2, y P2, z P2) in the camera 2, then the two coordinate must meet (8).

$$ \rho \left[\begin{array}{c}\hfill {X}_{P1}\hfill \\ {}\hfill {Y}_{P1}\hfill \\ {}\hfill 1\hfill \end{array}\right]={A}_1\left[{R}_{21},{t}_{21}\right]\left[\begin{array}{c}\hfill {x}_{P2}\hfill \\ {}\hfill {y}_{P2}\hfill \\ {}\hfill {z}_{P2}\hfill \\ {}\hfill 1\hfill \end{array}\right] $$
(8)

The matrix A1 represents the intrinsic parameters matrix of camera 1. [R 21 , t 21 ] represents the transformation from V 2 CF to V 1 CF. According to (8), if we get a series of image points in the camera1 and world points in the camera 2, we can obtain the outside parameter matrix [R 21 , t 21 ].

The following steps are used to complete global calibration:

  1. 1.

    According to Section 2, the positional relationship between the camera 2 and the light plane as shown in Fig. 2 can be determined.

  2. 2.

    Obtain the feature points in light plane by the chessboard. And camera 1 and 2 capture the images at the same time. Calculate the image coordinates (X P1 , Y P1 ) of the feature point P in the camera 1.

  3. 3.

    Calculate the image coordinates (X P2 , Y P2 ) of the feature point P in camera2.

  4. 4.

    According to the calibration results in step 1, calculate the world coordinates (x P2 , y P2 , z P2 ) of the feature point P in the camera 2.

  5. 5.

    According to the size of the chessboard, we can get multiple image plane coordinates in camera1and world coordinates in camera 2 at each position. Then by freely moving the target, we can get enough image coordinate points in camera1 and world coordinate points in camera 2. Put the coordinates into (8), using the L-M optimization algorithm [10], we can obtain [R 21 , t 21 ].Other V i FC (i = 3,4, ......) coordinate system can be transformed directly or indirectly through the above method. That is the global calibration method we proposed.

4 Experiment Result

The calibration results are shown as follows:

Camera 1: Intrinsic Matrx A 1

$$ {A}_1=\left[\begin{array}{ccc}\hfill 1605.638\hfill & \hfill 0\hfill & \hfill 640.945\hfill \\ {}\hfill 0\hfill & \hfill 1616.061\hfill & \hfill 509.606\hfill \\ {}\hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right] $$

The radial and tangent distortion coefficients:

k 1  = -9.77 × 10-2, p 1  = 5.20 × 10-2, k 2  = 1.29 × 10-3, p 2  = -2.67 × 10-3

Camer 2: Intrinsic Matrx A 2

$$ {A}_2=\left[\begin{array}{ccc}\hfill 1607.821\hfill & \hfill 0\hfill & \hfill 674.013\hfill \\ {}\hfill 0\hfill & \hfill 1617.634\hfill & \hfill 495.395\hfill \\ {}\hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right] $$

The radial and tangent distortion coefficients

k 1  = -8.65 × 10-2, p 1  = 4.13 × 10-2, k 2  = -2.17 × 10-3, p 2 = 1.16 × 10-3

Camer 3: Intrinsic Matrx A 3

$$ {A}_3=\left[\begin{array}{ccc}\hfill 1608.240\hfill & \hfill 0\hfill & \hfill 592.927\hfill \\ {}\hfill 0\hfill & \hfill 1618.168\hfill & \hfill 477.667\hfill \\ {}\hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right] $$

The radial and tangent distortion coefficients:

k 1  = -8.67 × 10-2, p 1  = 8.60 × 10-2, k 2  = -9.17 × 10-3, p 2  = -9.72 × 10-3

The equation of the light plane1 under coordinate V 2 CF:

$$ x\hbox{-} 0.017y-0.588z+293.375=0 $$

The equation of the light plane2 under coordinate V 3 CF:

$$ x\hbox{-} 0.002y-0.692z-324.375=0 $$

The Rotation and Translation Matrix [R 21 ,t 21 ]from V 2 CF to V 1 CF is:

$$ \left[{R}_{21},{\mathrm{t}}_{21}\right]=\left[\begin{array}{cccc}\hfill 0.561\hfill & \hfill \hbox{-} 0.0015\hfill & \hfill \hbox{-} 0.828\hfill & \hfill 427.411\hfill \\ {}\hfill \hbox{-} 0.002\hfill & \hfill 1\hfill & \hfill \hbox{-} 0.003\hfill & \hfill \hbox{-} 7.136\hfill \\ {}\hfill 0.829\hfill & \hfill 0.004\hfill & \hfill 0.561\hfill & \hfill 228.762\hfill \end{array}\right] $$

The Rotation and Translation Matrix [R31, t31] from V3CF to V1CF is:

$$ \left[{R}_{31},{\mathrm{t}}_{31}\right]=\left[\begin{array}{cccc}\hfill 0.303\hfill & \hfill 0.002\hfill & \hfill 0.953\hfill & \hfill 427.411\hfill \\ {}\hfill 0.026\hfill & \hfill 1\hfill & \hfill \hbox{-} 0.011\hfill & \hfill \hbox{-} 11.486\hfill \\ {}\hfill \hbox{-} 0.953\hfill & \hfill 0.028\hfill & \hfill 0.302\hfill & \hfill 319.533\hfill \end{array}\right] $$

An experiment to verify the accuracy of the calibration result is designed. Feature point P on light plane is selected. Camera 1 and 2 capture P at the same time. We can get P’s image coordinate (X P1 , Y P1 ) in camera 1.and P’s world coordinate(x P2 , y P2 , z P2 ) in camera 2 can be calculated according to the calibration result above. Equation (8) can make it possible to translate the coordinate (x P2 , y P2 , z P2 ) to camera 1’s image coordinate (X P1 ’, Y P1 ’). The D-value between (X P1 , Y P1 ) and (X P1 ’, Y P1 ’) could illustrate the precision of the calibration result. And the results are shown in Tables 1 and 2.

Table 1 Calibration precision between camer 1 and camera 2
Full size table
Table 2 Calibration precision between camer 1 and camera 3
Full size table

Ten feature points are obtained from light plane1 and 2. Calibration precision between camer1 and camera 2: the average reproject error in coordinate x is 0.014 pixel and -0.07 pixel in coordinate y. Calibration precision between camer 1 and camera 3: the average reproject error in coordinate x is 0.398 pixel and 0.476 pixel in coordinate y. The results of the experiment show that the method proposed above is feasible as well as high-precision.

5 Conclusions

The novel global calibration method for MVS proposed in this paper, which involved a structured light and plane target, provides a good accuracy. The method makes use of the uniqueness of the feature point to calculate the rotation and translations matrix between MVS. It is easy to use on-site with limited space. The target is easy to make with low-cost. What’s more, the real experiment results show a good performance of the proposed methods.