Keywords

1 Introduction

With the development of the color reproduction technology, people demand more and more accuracy on reproduction of color; broadband multispectral image acquisition technology can record the spectral reflectance of objects accurately and achieve the true color reproduction. Achieving the digital image through color digital camera is the most direct and effective way currently, and it can shoot the surface of the object of non-contact under the complex light. Therefore, it is the research hotspot based on digital camera multispectral image acquisition and spectral reflectance reconstruction in the field of color science currently.

Predecessors’ studies on the spectral reconstruction have put forward many shaping algorithms, such as Wiener estimation [1], the pseudo inverse method [2, 3], the finite dimension model [2, 4] and polynomial regression algorithm [4]. The selection methods of training samples include mainly Hardeberg method [5] and Cheung and Westland method [6], etc. Some of training samples are based on high dimensional spectral reflectance space, and others are based on chroma space. But each method only considered the spatial of spectral reflectance or chroma space, and has no comprehensive characteristics of both spectral reflectance space and chroma space. Therefore, spectral reconstruction precision is limited.

A method for combining principal component analysis with polynomial model was presented. The reconstruction results improved in removal of noise interference and improving the linearity. A method for spectral reflectance reconstruction based on sample optimization was also put forward. Sample optimization considers comprehensively the similarity of chroma space and broad representativeness of spectral spatial, and improves the accuracy of spectral reconstruction and chroma.

2 Basic Theory

2.1 Spectral Reconstruction Model Based on Principal Component Analysis and Polynomial Model

Spectral reflectance of natural object surface is continuous and can be represented as a linear combination of several basic vectors. Therefore, the spectral reflectance of the color can be reconstructed by a finite dimensional linear model, which can reduce the dimension of the prediction space effectively and reduce the complexity of the reconstruction process. The principal component analysis method can fully extract the spectral reflectance characteristics of the color samples, and it is typical representative of these methods and is applied to the reconstruction of the color spectrum of a variety of imaging devices successfully.

In practical application, it is generally believed that the spectral reflectance curve is smooth and spectral reflectance data set \( R_{0} \) of sample optimization can be expressed of linear combination of several main feature vectors \( F_{0} \) [7]. So the sample optimization spectral reflectance \( R_{0} \) can be expressed as:

$$ R_{0} = \sum\limits_{i = 1}^{l} {e_{i} \alpha_{i} = F_{0} \alpha_{0} } $$
(1)

where \( e_{i} \) is the feature vector, \( F_{0} \) is i-dimensional base vector of sample optimization, \( \alpha_{0} \) is the corresponding coefficient vector.

The relationship between response value of the camera and the coefficient vector \( \alpha_{0} \) can be obtained according to the polynomial principle [8]. Assuming that \( g_{0} \) represents a polynomial matrix which is consisting of N vectors \( [R_{0} \;G_{0} \;B_{0} ] \), then there is a matrix M that meets the following relationship:

$$ \alpha_{0} = Mg_{0} $$
(2)

The transformation matrix M can be obtained by the following formula based on the least square method:

$$ M = \alpha_{0} (g{_{0}}^{\text{T}} g_{0} )^{ - 1} g{_{0}}^{\text{T}} $$
(3)

\( g_{0} \) is a polynomial model of the sample optimization camera response value. The transformation matrix will change when the number increasing of \( g_{0} \) items. Precision and items of polynomial model are closely related to its extension type.

Then the reconstructed spectral reflectance is:

$$ \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{R}_{{}} = \hat{R}_{0} g_{0}^{ + } g $$
(4)

where \( \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{R} \) is the spectral reflectance of sample to be reconstructed; \( \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{R}_{0} \) is the spectral reflectance estimation of sample optimization; \( g_{0}^{ + } \) and g are pseudo inverse matrix of polynomial model of sample optimization camera response value and polynomial model of reconstructed sample camera response value.

2.2 Theory and Method on Sample Optimization

Spectral reflectance reconstruction by training samples is also known as the reconstruction method based on learning, which can make full use of sample optimization information, improve the accuracy and stability of the regression function, have better generalization performance, and avoid usual “over-fitting” problem in regression analysis brought by too many samples. So the training sample is an important factor to affect spectral reflectance reconstruction. Literature [9] demonstrated when training samples reach a certain amount for color order system with numerous samples. It will produce redundancy and the reconstruction results decrease if the color sample continues increasing. Therefore, we need optimize sample from all the samples, and make them present spectral spatial and chroma space characteristics.

In this paper, the principle of sample optimization is responding of characteristics of color order system by fewer samples, and has broad representativeness of spectral reflectance space and the similarity of chroma space. The optimized method and steps are as follows:

  1. (1)

    In Spectral reflectance space, training samples are correlated with one another as small as possible and we define them as the set of samples \( A = (\alpha_{1} ,\alpha_{2}\, \ldots \ldots\, \alpha_{n} ) \). The first sample of set A selects modulus maximum of spectral reflectance vector and standard of selecting is minimum correlation matrix between the accession of each training vector spectral reflectance vector and the original set from second samples, then the nth selection of training sample satisfies the Eq. (5):

    $$ \begin{aligned} & \frac{{\omega_{ \hbox{max} } ([R_{{a_{1} }} ,R_{{a_{2} }} , \ldots\, R_{{_{{a_{n} }} }} ])}}{{\omega_{ \hbox{min} } ([R_{{a_{1} }} ,R_{{a_{2} }} , \ldots\, R_{{_{{a_{n} }} }} ])}} \le \frac{{\omega_{ \hbox{max} } ([R_{{a_{1} }} ,R_{{a_{2} }} , \ldots\, R_{{a_{n - 1} }} ,R_{i} ])}}{{\omega_{ \hbox{min} } ([R_{{a_{1} }} ,R_{{a_{2} }} , \ldots\, R_{{a_{n - 1} }} ,R_{i} ])}} \\ & \quad 1 \le i \le T,\quad i \notin \{ a_{1} ,a_{2} , \ldots, a_{n - 1} \} \\ \end{aligned} $$
    (5)
  2. (2)

    We do cluster according to the K-means clustering method in chroma space of CIELAB with the i classification initial point of condensation which is chroma coordinate \( (L^{*} ,a^{*} ,b^{*} ) \) of \( \alpha_{i} \) in set A, because difference between samples could be distinguished with Euclidean distance in uniform chroma space. After clustering, each subset is clustering samples with similar characteristics, and could reflect the specific chroma space features.

  3. (3)

    Clustering center was calculated for each class, and then sample optimization set is the nearest cluster center sample of each cluster in color order system according to the clustering results.

    Sample optimization according to above-mentioned method is clustering result for color order system samples in chroma space. Clustering center can represent the main characteristics information of such class according to the similarity theory, so it has the maximum effect on spectral reflection reconstruction if the sample is the nearest to cluster center.

3 Experiments and Result Analysis

In the experiment, we used GTI MiniMatcher standard light box and D65 standard light source; Nikon D7000 digital camera, ColorChecker SG standard color card (abbreviation for SG card) 140 colors; RAW format; recorded RGB value of SG card. It shot the dark current image after obtaining the each image through closing the shutter. Spectral reflectance of SG color card has been measured under the field of 2° view, D65 light source by the Spectrolino.

As mentioned above, the spectral reconstruction process is divided into the following steps: (1) the color card was optimized and got 20 optimized samples after obtaining experimental data; (2) the principal component analysis was performed on the spectral reflectance of sample optimization, base vector matrix \( F_{0} \) was obtained and the corresponding coefficient vector \( \alpha_{0} \). (3) Polynomial model was used to determine the relationship between the response value \( R_{0} \;G_{0} \;B_{0} \) of the camera and coefficient vector \( \alpha_{0} \) of the sample optimization; (4) Spectral reflectance was reconstructed according to the formula (4).

3.1 Optimized Sample and Its Characteristics

We obtained optimization sample sets in SG color card adopting above-mentioned method. Figure 1 is distribution chart in chroma space and principal component space. As can be seen from Fig. 1, sample optimization distribution is more well-distributed in chroma space, and its distribution of Hardeberg method is more centralized relatively, and Cheung and Westland method are mostly distributed on the edge. In the principal component space, Hardeberg method distributes well and Cheung & Westland method are more centralized relatively. So the spectral reflection reconstruction is prone to “over-fitting” phenomenon. Overall distribution of sample optimization is more well-distributed although there are near samples, so the spectral reflection reconstruction can achieve good effect.

Fig. 1
figure 1

Sample distribution chart of chroma space and principal component space for sample optimization, Hardeberg method and Cheung and Westland method

Full size image

3.2 Determination of Sample Optimization Base Vector

In order to determine the relationship between the polynomial model of the camera response value and the coefficient vector \( \alpha_{0} \), the principal component analysis is performed on the spectral reflectance of sample optimization. As shown in Table 1, the cumulative contribution rate reached 98.85% when the number of principal components is 9. So we selected 9 basis vectors and its corresponding coefficient vectors considering the calculation and other factors.

Table 1 The number of principal components and their cumulative contribution rate
Full size table

3.3 Establishment of the Relationship Matrix Between the Camera Response Value of Sample Optimization and the Principal Component Coefficients Vector According to Polynomial Model

The polynomial model could generate high-dimensional input data through selecting the combination item of camera response value \( g_{0} = [R_{0} \;G_{0} \;B_{0} ] \) with a group of equations. These data through linear combination are fitting for coefficient vector \( \alpha_{0} \) of sample optimization and then the coefficient of equation group is obtained. After many experiments, the final polynomial model of sample optimization \( g_{0} \) is:

$$ g_{0} = [R{_{0}}^{2} \;G{_{0}}^{2} \;B{_{0}}^{2} \;R_{0} \;G_{0} \;G_{0} \;B_{0} \;R_{0} \;B_{0} \;R_{0} \;G_{0} \;B_{0} ]. $$
(6)

3.4 Spectral Reflectance Reconstruction and Its Evaluation

Spectral reflectance of SG color card was reconstructed on different sample selection method according to the principal component analysis, polynomial model (6) and formula (4). Experimental results were shown in Table 2.

Table 2 Reconstruction results comparison of mean of RMSE, mean of GFC and average CIE DE2000 for SG color card on Hardeberg, Cheung and Westland and optimized methods
Full size table

Table 2 shows that principal component analysis combined with polynomial model and sample optimization method are better than literature method on effect on spectral reflection reconstruction for SG color card and spectral accuracy and chroma accuracy are improved.

Figure 2 is the spectral reflectance curve of the best sample and worst sample for spectral reconstruction effect.

Fig. 2
figure 2

Reconstruction spectral reflectance curve of the best sample and worst sample for SG color card

Full size image

4 Conclusions

It can accurately simulate the nonlinear relationship between camera response value and coefficient vectors by using principal component analysis combined with the polynomial model method and the spectral reflectance of the object surface can be reconstructed accurately. Sample optimization method has not only broad representatives of spectral space, but also the similarity of chroma space, avoiding “over-fitting” phenomenon brought by excessive samples. The accuracy of spectral reconstruction is good. The mean of RMSE and the average color difference are improved obviously in spectral reflection reconstruction through sample optimization. The mean of GFC is increased which are better than selection methods of traditional sample. It could better meet the requirements of high precision color duplication of production and life, art digital collection, color multimedia display and color quality evaluation etc.