Keywords

1 Introduction

Spectral sensitivities represent the spectral response of a camera at different wavelength. It can be applied in multispectral imaging, colour constancy, spectral reflectance recovery et al. [1,2,3]. However, this information is generally not provided by the manufacturer. The standard method of camera spectral calibration is to use a monochromator [4, 5]. This device can generate monochromatic lights at different wavelength, and the camera is used to capture a series of images over the desired range of wavelength. But it is a costly device and has not been widely applied. Various algorithms have been applied to estimate the camera spectral sensitivities by capturing colour samples. The colour samples mainly included reflective colour charts [6, 7], LED-based samples [8], fluorescence [9] and a display [10]. Among different media of colour samples, a display was the most common device in our daily life. In the study of Zhu et al. [10], 81 colour patches designed from orthogonal test were presented on a LCD display, and the camera spectral sensitivities were estimated by window filters on frequency and spatial domains.

In this study, the principal component analysis (PCA) algorithm was implemented with colour samples presented on a display. Our hypothesis is that accurate estimation of camera spectral sensitivities can be achieved by employing the three primaries (red, green and blue) of a display and their combinations (white, cyan, magenta and yellow) as training samples.

2 Method

The linear responses of a camera can be formulated by Eq. (1), where \(X_{i}\) is the camera response, \(r\left( \lambda \right)\) is the spectral reflectance of object, \(e\left( \lambda \right)\) is the spectral power distribution (SPD) of the light, \(s_{i} \left( \lambda \right)\) is the spectral sensitivity of the camera, and [\(\lambda_{min} \lambda_{max}\)] is the range of visible spectrum. For a common trichromatic camera, there are three channels corresponding to R, G and B.

$$ X_{i} = \int\limits_{{\lambda_{min} }}^{{\lambda_{max} }} {r\left( \lambda \right)e\left( \lambda \right)s_{i} \left( \lambda \right)d\lambda ,\,\,i = R,G,B} $$
(1)

After sampling the spectrum from 400 nm to 700 nm with 10 nm interval, the above equation can be written in matrix donation as Eq. (2), where \(X\) is the N by 3 matrix of camera responses, r is the N by 31 spectral reflectance of objects, E is the 31 by 31 diagonal matrix indicating the SPD of light, and S is the 31 by 3 camera spectral sensitivities. N is the number of the objects.

$$ X = rES $$
(2)

Suppose L to be the spectral signals received by the camera. It is the product of the spectral reflectance of objects and the SPD of light. Then Eq. (2) can be written as Eq. (3),

$$ X = LS $$
(3)

In general, the estimation of camera spectral sensitivities requires the known spectral signals L and the corresponding camera responses X.

In the study of Jiang et al. [11], spectral sensitivities database including 28 cameras was collected. They performed PCA on the spectral sensitivities and found the space was two-dimensional. In other words, the spectral sensitivities could be expressed by the first two eigenvectors as Eq. (4), where \(B_{i}\) is the eigenvector matrix and \(a_{i}\) is the coefficients.

$$ S_{i} = B_{i} a_{i} ,\,\,i = R,G,B $$
(4)

Substituting Eq. (4) into Eq. (3), then the least squares solution of \(S_{i}\) can be found by Eq. (5). The superscripts T and −1 denote respectively the matrix transpose and inverse.

$$ min_{{a_{i} }} \left\| {{LB_{i} a_{i} - X_{i} }} \right\| ^{2} $$
$$ S_{i} = B_{i} \left[ {\left( {LB_{i} } \right)^{T} \left( {LB_{i} } \right)} \right]^{ - 1} \left( {LB_{i} } \right)^{T} X_{i} ,\,\,i = R,G,B $$
(5)

Considering that the dimensionality of spectral sensitivities has been decreased to only two by means of PCA, the colour samples with more than two dimensionalities can be used to recover the spectral sensitivities theoretically. As a result, the traditional trichromatic display that contains RGB primaries could be used as the target. In this study, the colour samples were presented on a display, including the primaries (red, green and blue) of a display and their combinations (white, cyan, magenta and yellow). These colours were divided into two groups. One includes red, green, blue and white, marked as RGBW, while the other is cyan, magenta, yellow and white, marked as CMYW. Figure 1 shows the patterns of colour samples presented on the display. By capturing the colour samples using a given camera, and measuring the corresponding SPD, it is expected that the spectral sensitivities of the camera can be estimated by PCA algorithm following Eq. (5).

Fig. 1.
figure 1

Patterns of colour samples presented on a display

Full size image

3 Experiment

The experiment was conducted in a dark room. Figure 2 shows the experimental situation. The patterns of colour samples as shown in Fig. 1 were presented on the mobile display of OPPO FindX3 Pro. It was a 6.7-inch OLED display with 3216 × 1440 pixels. A Canon 650D digital camera was used to capture the colour patterns. It should be noted that the camera of this model was not included in the spectral sensitivities database used for PCA. The setup of the camera (ISO, shutter speed, F-number) was adjusted to obtain appropriate exposure. The responses of the colour patches were extracted from the RAW images. An image under dark condition was also captured to subtract the dark noise from camera responses.

Fig. 2.
figure 2

Experimental situation of capturing the colour samples on the display

Full size image

The SPD of each colour patch was measured using a JETI-Specbos 1211 spectroradiometer. Figure 3 shows the SPD of the colour samples. Figure 4 shows the chromaticity in CIE 1976 u′v′ plane.

Fig. 3.
figure 3

Spectral power distribution (SPD) of the colour samples

Full size image
Fig. 4.
figure 4

Chromaticity of the colour samples in CIE 1976 u′v′ plane

Full size image

In addition, a reflective colour chart was captured to test the accuracy of the estimated spectral sensitivities. An Xrite Macbeth ColorChecker chart (MCCC) was placed in a Thouslite® LED viewing cabinet. The test lights were simulated D50, D65 and A. The camera was used to capture the chart with a viewing/illumination geometry of 0/45°. The SPD of the light was measured using the JETI-Specbos 1211 spectroradiometer. The spectral reflectance of the colour patches was measured using a Datacolor SF600 spectrophotometer. The spectral signals reflected by the colour chart were calculated as the product of the SPD of light and the spectral reflectance of the colour patches.

4 Results and Discussion

The spectral sensitivities of the camera were estimated from the two groups of display-based colour samples, RGBW and CMYW. And the same PCA algorithm was also applied to the 24 colour samples on MCCC under D65 to obtain the estimated spectral sensitivities. The ground truth spectral sensitivities were calibrated by a Labsphere QES1000 monochromator. Figure 5 plotted the comparison of the estimated spectral sensitivities from different colour samples, and the results were compared with the ground truth results by monochromator. It can be seen that all the three estimated spectral sensitivities had a close match with those calibrated by the monochromator. Differences rarely exist between the shapes of the three estimated results.

Fig. 5.
figure 5

Comparison of the spectral sensitivities calibrated by monochromator and those estimated from (a) RGBW samples on display, (b) CMYW samples on display, (c) MCCC under D65.

Full size image

Four different metrics were used to test the accuracy of the estimated spectral sensitivities, including spectral error (SE), RGB error, colour difference \({\Delta }E_{ab}^{*}\) and Vora value. These metrics were also used in the study of Finlayson et al. [6]. Let us denote the estimated and the ground truth spectral sensitivities as \(\hat{S}\) and S, respectively. The spectral error (SE) can be calculated as Eq. (6), indicating the spectral accuracy of the estimated spectral sensitivities compared with monochromator.

$$ SE = \frac{{\left\| {\hat{S} - S} \right\|}}{\left\| S \right\|} \times 100\% $$
(6)

The percentage RGB error was calculated between the measured RGB and the predicted RGB by spectral sensitivities. The predicted RGB is calculated by Eq. (2). Let \(\widehat{{X_{i} }}\) and \(X_{i}\) donate the predicted and measured camera responses of the sample i. The percentage RGB error can be calculated by Eq. (7), where N is the number of the testing samples. In this study, the testing samples were the 24 colours on MCCC under D50 and A.

$$ {\Delta }X{\text{\% }} = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \frac{{\left| {\widehat{{X_{i} }} - X_{i} } \right|}}{{\left| {X_{i} } \right|}} \times 100\% ,\,\,\,{\text{X}} = {\text{R}},{\text{G}},{\text{B}} $$
(7)

The colour difference \({\Delta }E_{ab}^{*}\) is calculated as the Euclidean distance in CIE 1976 L*a*b* colour space. Firstly, the linear color correction matrix (CCM) between camera responses and CIE tristimulus XYZ values under CIE D65 is developed. The camera responses are calculated via ground truth spectral sensitivities. The testing samples are the Leeds 100876 reflectance dataset [12]. Then the RGB values predicted by both the estimated and the ground truth spectral sensitivities are transformed to XYZ values and then to L*a*b*. The \({\Delta }E_{ab}^{*}\) between them is calculated to indicate the accuracy of estimated spectral sensitivities.

Vora value [13] is defined by Eq. (8). Its value ranges between 0 and 1. Larger Vora value means a closer match between \(\hat{S}\) and S.

$$ Vora = \frac{1}{3}trace\left( {S\left( {S^{T} S} \right)^{ - 1} S^{T} \hat{S}\left( {\hat{S}^{T} \hat{S}} \right)^{ - 1} \hat{S}^{T} } \right) $$
(8)

Table 1 lists the accuracy of the estimated spectral sensitivities in terms of different metrics. The best result of each metric was marked in bold. It can be found that the spectral sensitivities estimated from CMYW samples based on display had the highest accuracy in terms of SE, RGB error and Vora value, and slightly larger \({\Delta }E_{ab}^{*}\) value. However, the variations between different samples were not significant. Overall, it was concluded that using display-based samples to estimate the spectral sensitivities of a camera by PCA method could result in similar accuracy compared with that using a reflective colour chart MCCC.

Table 1. Accuracy of the estimated spectral sensitivities from different colour samples
Full size table

The advantage of using a display to present colour samples is that it is quite convenient and simple to use. It only needs a single shoot, so that it has higher efficiency compared with the method using a monochromator or multi-channel LED. The method includes only a spectroradiometer and a display. A display, especially a mobile display, is a common device in our daily life. In the future work, it will verify whether displays of different primaries will give similar spectral sensitivity functions and compare with their performance.

Nevertheless, using colour samples based on a display had its limitations. Due to the low dimensionality of display samples (usually only three for a trichromatic display with RGB primaries), only PCA algorithm could be applied because it considered the spectral sensitivities to be two-dimensional. So its accuracy was restricted by the PCA algorithm and the present spectral sensitivities database, and thus this method could only be applied to a trichromatic camera. In addition, the setup of the camera should be carefully adjusted to prevent the Moire fringe in the image when capturing the display. With the demand of higher estimated accuracy, a reflective colour chart or LED-based samples could be used together with other algorithms such as regularization and quadratic programming [5].

5 Conclusion

In this study, the spectral sensitivities of a camera were estimated by capturing colour samples presented on a display using PCA algorithm. The colour samples were arranged as RGBW and CMYW patterns respectively. The results showed the estimated spectral sensitivities to give a close match with those calibrated by a monochromator. The accuracy was evaluated in terms of spectral error, RGB error, colour difference and Vora value metrics. The results showed that the display-based method could have similar accuracy compared with the method based on a reflective colour chart MCCC.