1 Introduction

Digital projectors are becoming increasingly important in some fields, with high accuracy requirements such as three-dimensional (3D) printing, non-contact 3D measurement, and reality-guided systems. Because of the non-linear distortion and assembly error of the projection lens, the accuracy of these systems is highly variable. Therefore, it is necessary to calibrate and correct the non-linear distortion of a digital projector, especially in accuracy-sensitive applications. A projector has a reverse optical system with respect to a camera. The linear model (pinhole model) and the non-linear distortion model of the imaging optical system can be also used in a projecting optical system. Like the camera, many calibration methods have been developed to obtain the intrinsic and distortion parameters for the projector. Drareni et al. proposed a basic method for calibrating a projector using the homography between projector and camera image planes [1]. Radu et al. presented a similar method that relies on calculating vanishing points [2]. In the literature [3–5], a classical projector calibration method was presented that is based on virtual feature extraction (the non-linear projecting distortion is considered). To prevent the calibration error caused by camera distortion, many methods based on spatial phase analysis have also been developed [6–8]. There are also some implicit calibration methods that treat the projector and camera as an integrated system [9, 10] to obtain more precise measurement results. Except for the methods based on polynomial distortion representation [9, 11] or look-up table representation [10], nearly all the calibration methods generate the intrinsic and distortion parameters of the projector. In some 3D measurement applications, the parameters can be used to correct the measurement result directly; in other applications, the image to be projected should be rectified by the calibration result to obtain a corrected pattern on the projector object plane. Therefore, a precise full-field image rectifying procedure using the intrinsic and distortion parameters should be applied to the original image before projecting it. The existing image rectifying methods designed for cameras cannot be used to process a projected image because of the reverse imaging structure of the projector. Thus, the image rectifying procedure for a projector should be rearranged by analyzing the corresponding projector calibration procedure.

Moreover, the existing evaluation methods for projector distortion calibration cannot give a complete evaluation result. The most commonly used method is to calculate the reprojection error after optimizing the intrinsic and distortion parameters [4–7]. This is a general method to evaluate the calibration data processing procedure and optimizing algorithm for projector calibration, but the errors caused by feature extraction, phase calculation, and other factors are ignored. In a 3D projection measurement system, measuring accuracy can be treated as a combined evaluation method (e.g., planar plate evaluation [12] or ball surface evaluation [13]). However, the system evaluation cannot provide an evaluation independently, so it is not suitable for an application that only uses a projector (e.g., a 3D painting application).

In this paper, a precise full-field rectifying technique is proposed to compensate for the distortion of the projection lens. The generic procedure of rectifying a pre-projecting image is derived by analyzing the lens distortion representation and corresponding calibration method. Furthermore, a complete evaluation method based on spatial phase information is presented to evaluate the effectiveness of the projector calibration. The homography between the projector image plane and camera image plane can be precisely calculated with a planar calibration plate. Based on this homography, a full-field non-linear distortion distribution map can be generated to represent the distortion characteristics of nearly all the projector’s pixels.

The rest of the paper is organized as follows. In Sect. 2, the full-field image rectifying procedure and corresponding verification method for a digital projector are presented. In Sect. 3, the experiment system configuration and calibration procedure of the projection lens are presented, and the final calibration accuracy is verified by using the proposed full-field evaluation method. Finally, Sect. 4 concludes the paper.

2 Principle

2.1 From projector calibration to image rectification

Like a camera, the simplified linear model (pinhole model) can be used to describe an ideal projector. The relationship between pixel \(P_{p} \left( {u_{p} ,v_{p} } \right)\) on the projector image plane and point \(P\left( {x,y,z} \right)\) in the projecting space can be expressed as

$${\text{s}}\left[ {\begin{array}{*{20}c} {u_{p} } \\ {v_{p} } \\ 1 \\ \end{array} } \right] = M_{p} \left[ {\begin{array}{*{20}c} R & T \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} x \\ y \\ z \\ \end{array} } \\ 1 \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {f_{pu} } & 0 & {c_{pu} } \\ 0 & {f_{pv} } & {c_{pv} } \\ 0 & 0 & 1 \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {r_{11} } & {r_{12} } & {r_{13} } \\ {r_{21} } & {r_{22} } & {r_{23} } \\ {r_{31} } & {r_{32} } & {r_{33} } \\ \end{array} \begin{array}{*{20}c} {t_{1} } \\ {t_{2} } \\ {t_{3} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} x \\ y \\ z \\ \end{array} } \\ 1 \\ \end{array} } \right],$$
(1)

where \(M_{p}\) is the intrinsic parameters matrix of the projector,\(\left( {c_{pu} ,c_{pv} } \right)\) are the coordinates of the principal point in the projector image coordinate system, \(f_{pu}\) and \(f_{pv}\) are the equivalent focal length along the two orthogonal directions defined by the projector image plane, \(R\) and \(T\) are, respectively the rotation matrix and translation vector containing the extrinsic parameters, and \(s\) is a scale factor.

To apply the two-step calibration method [14] to the projector, one point coordinate \(\left( {x_{cp} ,y_{cp} ,0} \right)\) on a spatial plane and the corresponding point coordinate \(\left( {u_{p} ,v_{p} } \right)\) on the projector image plane should be obtained to find the homography \((H)\) between the calibration plate plane and projector image plane in one image. To find the optimized estimate of \(M_{p}\), the close-form solution mentioned in [15] can be used to establish several equations. Minimizing the reprojection error of each marker point on the projector image plane is a nonlinear optimization problem that can be solved by the Levenberg–Marquardt algorithm.

The nonlinear model of a projector uses radial and tangential distortion coefficients to describe the lens distortion. The following equations give the relationship from point \(\left( {u_{p} ,v_{p} } \right)\) on the projector image plane to the corresponding point \(\left( {u_{p\_d} ,v_{p\_d} } \right)\) in the projected ideal pattern on the projector object plane. Only the first two terms of radial and tangential distortion are considered in the following equations:

$$x_{p\_d} = x_{p} \left( {1 + k_{1} r^{2} + k_{2} r^{4} } \right) + \left[ {2p_{1} x_{p} y_{p} + p_{2} \left( {r^{2} + 2x_{p}^{2} } \right)} \right],$$
(2)
$$y_{p\_d} = y_{p} \left( {1 + k_{1} r^{2} + k_{2} r^{4} } \right) + \left[ {p_{1} \left( {r^{2} + 2y_{p}^{2} } \right) + 2p_{2} x_{p} y_{p} } \right],$$
(3)

where

$$x_{p} = \left( {u_{p} - c_{pu} } \right)/f_{pu} ,$$
(4)
$$y_{p} = \left( {v_{p} - c_{pv} } \right)/f_{pv} ,$$
(5)
$$r^{2} = x_{p}^{2} + y_{p}^{2} .$$
(6)

Distorted point \(\left( {u_{p\_d} ,v_{p\_d} } \right)\) can be calculated as

$$u_{p\_d} = f_{pu} { \cdot }x_{p\_d} + c_{pu} ,$$
(7)
$$v_{p\_d} = f_{pv} { \cdot }y_{p\_d} + c_{pv} .$$
(8)

Point \(\left( {u_{p\_d} ,v_{p\_d} } \right)\) takes the place of \(\left( {u_{p} ,v_{p} } \right)\) in Eq. (1). Disregarding distortion and using the calculated \(M_{p}\) for the initial values, the intrinsic parameters \(M_{p\_d}\) are calculated again with a nonlinear model of the projector. Meanwhile, the distortion parameters \(k_{1} ,k_{2} ,p_{1} ,{\text{and }}p_{2}\) are acquired, and these will further minimize the reprojection error of the marker points.

Note that the nonlinear model is applied to correct the linear model using the distortion parameters in the calibration procedure. The calibration results, optimized with the nonlinear model, will lead to the optimal approximation to the ideal corner position that should be projected. In other words, if a generated point is at the corrected position calculated by Eqs. (2)–(8), the projected point will be at the ideal position calculated by Eq. (1). Therefore, the distortion parameters \(k_{1} ,k_{2} ,p_{1} ,{\text{and }} p_{2}\) and Eqs. (2)–(8) can be used to calculate the corrected pixel position of the projector to compensate for the distortion. In conclusion, to correct a single point on the original image, Eqs. (2)–(8) can be used to calculate the subpixel coordinates on the rectified image. To rectify a full-field image, a remapping-interpolating process should be applied to each pixel of the projector. The whole procedure is illustrated in Fig. 1, where N um is the total number of pixels in the original image.

Fig. 1
figure 1

Flow chart of the procedure to rectify a full-field image

Full size image

2.2 Full-field evaluation method based on spatial phase information

To be independent of the calibration process, an evaluation method based on spatial phase information [16] is proposed to replace the existing corner extraction methods. Another advantage of using a spatial phase calculation technique is that the distortion characteristics of almost all of the projector’s pixels can be obtained. The evaluation method can be illustrated by the flow chart shown in Fig. 2.

Fig. 2
figure 2

Flow chart of the evaluation method

Full size image

The key process of the proposed evaluation method is to map all the pixels of the projected pattern back to the original image on the projector image plane. A calibrated camera and a white calibration plate with small concentric ring markers are used to achieve the objective. The mapping process can be achieved by analyzing the ring markers and phase information on the captured image.

It is necessary to determine the coordinate transform from the projector image plane to the camera image plane, which is defined as homography \(H_{p}^{c}\). The calibration plate acts as an intermediary for calculating \(H_{p}^{c}\). With the known coordinates of the marker points on the calibration plate and the marker points’ image (corrected by the camera’s distortion parameters), homography \(H_{cp}^{c}\) from the calibration plate to camera can be calculated.

To establish the relationship between the projector and calibration plate, the spatial phase calculation technique is applied because the phase value can be mapped to the projector image plane with sub-pixel accuracy. First, a set of ideal horizontal and vertical phase-shifted fringe patterns are generated and projected onto the calibration plate. The absolute phase data is calculated from the projected fringe patterns. All centers of the ring markers can be extracted by an ellipse fitting method, and the absolute phase \(\left( {\varphi_{pu} ,\varphi_{pv} } \right)\) in two directions corresponding to the center of a specific ring marker can be calculated [7]. Second, corresponding coordinates \(\left( {u_{p} ,v_{p} } \right)\) on the projector image plane can be calculated from absolute phase \(\left( {\varphi_{pu} ,\varphi_{pv} } \right)\) through linear interpolation. Third, homography \(H_{cp}^{p}\) from the calibration plate to the projector can be calculated with the known position of ring markers on the calibration plate and corresponding point \(\left( {u_{p} ,v_{p} } \right)\) on the projector image plane. Finally, homography \(H_{p}^{c}\) from the projector to the camera can be obtained using the following equation:

$$H_{p}^{c} = H_{cp}^{c} { \cdot }\left( {H_{cp}^{p} } \right)^{ - 1} .$$
(9)

All of the projector pixel coordinates on the original image can be mapped to the camera image plane in a linear model with the obtained \(H_{p}^{c}\). The coordinates on the camera image plane should be distorted by the distortion parameters of the camera to find the phase on the phase map that is distorted by the camera’s imaging optical system. Comparing one point’s phase on the original image and the corresponding point’s phase on the calculated phase map along horizontal and vertical directions, the deviation of each projector pixel can be obtained.

To validate the corrected results of the calibration, the pre-distorted image is projected onto the calibration plate and the final result can be evaluated by calculating the deviation between the corrected back projection image and the original image.

3 Experiments and results

3.1 Projector calibration

To validate the proposed full-field rectification technique for projector distortion correction, a projector calibration system was built. The system includes a DLP® projector, camera, and calibration plate with hollow ring markers on the surface, as illustrated in Fig. 3. The projector is a Lightcrafter 4500 from Texas Instruments with a resolution of 912 × 1140. The projecting field width was adjusted to 250 mm. The camera is a B3320M-8MP from Imperx with a 25 mm prime lens. The calibration method detailed in Sect. 2.1 was used to calculate the intrinsic matrix and distortion parameters of the projector. The calibration procedure followed the method detailed in [4]. However, to improve the credibility of the calibration results, two details of the procedure were changed. First, a diamond array pattern was used to take the place of the checkerboard pattern because the projector (Lightcrafter 4500) for calibration is equipped with diamond-shaped pixels. Second, the white paper used to cover the original pattern on the calibration plate was discarded. The corner points coinciding with the printed ring markers were all removed before the parameters of the projector were calculated. The subpixel corners were extracted [17] and then the parameters were calculated. The best estimate of the intrinsic matrix and distortion parameters of the projector are listed in Table 1 [18, 19] .

Fig. 3
figure 3

Calibration and evaluation system for a digital projector. a Calibration plate, b digital projector, and c camera

Full size image
Table 1 Intrinsic parameters of the projector
Full size table

The reprojection error of the calibrated projector is illustrated in Fig. 4.

Fig. 4
figure 4

Reprojection error of projector (unit: pixels)

Full size image

The mean value and standard deviation of reprojection error are 0.1204 pixels and 0.1470 pixels, respectively.

3.2 Full-field evaluation for projector calibration

A control experiment was carried out to evaluate the quality of the calibrated projector. The projector and camera were moved to another position to obtain the validation result. The original and corrected phase-shifting fringe patterns were projected onto the calibration plate in the proper sequence. The procedure of the evaluation experiment follows the method described in Sect. 2.2.

After calculating the absolute phase maps from the captured fringe pattern images, the actual back projection error map of the original patterns and corrected patterns were calculated, as shown in Figs. 5 and 6. The error distribution does not have central symmetry because the projector has a 100 % optical offset in the Y direction. The holes on the surface denote the “low contrast” pixels covered by the ring markers on the calibration plate. These points are dropped from the final error map.

Fig. 5
figure 5

Actual back projection error map of the original pattern (unit: pixels)

Full size image
Fig. 6
figure 6

Actual back projection error map of the corrected pattern (unit: pixels)

Full size image

Before rectification, the average value and standard deviation of the actual back projection error were 0.6690 pixels and 1.0573 pixels, respectively. After rectifying the projected patterns using the non-linear distortion obtained using the proposed method, they were 0.1470 pixels and 0.1695 pixels.

The experimental results show that the full-field rectification technique and evaluation method for projector distortion are correct and effective. Of course, some areas with accuracy deterioration still exist. Two factors can explain this phenomenon. First, the distortion characteristics cannot be perfectly described by radial and tangential distortion parameters. Second, no corner points are generated near the edge of the projector image plane in the calibration procedure that is described in Sect. 3.1.

4 Conclusions

This paper proposes a new full-field distortion correcting technique and corresponding evaluation method for digital projectors. By analyzing the calibration procedure of the projector, an image rectifying technique was proposed using the radial and tangential distortion representations. To validate the proposed technique, a full-field evaluation method was presented based on spatial phase information. The experimental results show that the image rectification technique and the corresponding evaluation method for projector distortion correction are correct and effective. Furthermore, the proposed evaluation method can be used as a generic way to evaluate the non-linear distortion characteristics of a projector. The distortion characteristics can be revealed by a full-field error distribution map precisely and intuitively.