On the von Kries Model: Estimation, Dependence on Light and Device, and Applications

Abstract

The von Kries model is widely employed to describe the color variation between two pictures portraying the same scene but captured under two different lights. Simple but effective, this model has been proved to be a good approximation of such a color variation and it underpins several color constancy algorithms. Here we present three recent research results: an efficient histogram-based method to estimate the parameters of the von Kries model, and two theoretical advances, that clarify the dependency of these parameters on the physical cues of the varied lights and on the photometric properties of the camera used for the acquisition. We illustrate many applications of these results: color correction, illuminant invariant image retrieval, estimation of color temperature and intensity of a light, and photometric characterization of a device. We also include a wide set of experiments carried out on public datasets, in order to allow the reproducibility and the verification of the results, and to enable further comparisons with other approaches.

An erratum to this chapter is available at 10.1007/978-94-007-7584-8_14

An erratum to this chapter can be found at http://dx.doi.org/10.1007/978-94-007-7584-8_14

1 Introduction

Color is one of the most important features in many Computer Vision fields such as image retrieval and indexing [48], object and scene recognition [53], image segmentation [49], and object tracking [39]. Although color is robust to many image geometric distortions, e.g. changes of image size and/or orientation, and to noise, its use in practical applications is often limited by its strong sensitivity to the light. In fact, we experience that the same scene viewed under two different lights produces two different pictures. This is because the color of an image depends on the spectral profile of the light illuminating the scene, on the spectral reflectivity and geometry of the materials composing the scene, and on the device used for the acquisition. Features like colors, luminance, magnitude and orientation of the edges, which are commonly used to describe the visual appearance of a scene, remarkably change when the light varies. As a consequence, many algorithms for image classification and/or object recognition, that are based on these features (e.g. [38, 40]), do not work in case of illuminant variations [53]. Understanding how the colors of a picture vary across the illumination of the imaged scene is a crucial task to develop a recognition and/or retrieval system insensitive to light conditions.

In the human visual system, the color illuminant invariance is achieved by a chromatic adaptation mechanism named color constancy: it detects and removes possible chromatic dominants and illuminant incidents from the observed scene, so that the same scene under different illuminants is perceived as the same entity [54]. Although color constancy has been intensively investigated in the past decades, it remains still an unsolved problem [10, 24]. In the last years, many methods for simulating this human capability have been developed [2, 21–23, 25, 29, 33]. A recent survey on the main approaches is presented in [28], while a comparison of the most used color constancy algorithms can be found in [5, 6]. Advantages and disadvantages in using some of these methods are addressed in [1].

The von Kries model is widely used to describe the color variation between images or image regions due to an illuminant change. It relies on three main assumptions: (i) the lights under which the images (or regions) are acquired are spatially uniform across the scene; (ii) the materials composing the imaged scene are Lambertian; (iii) the device used for capturing the images is narrow-band, or the spectral sensitivities of its sensors do not overlap. Hypothesis (i) constraints the spectral power distribution of the light to be homogeneous across the scene. Hypothesis (ii) holds for a lot of wide matte materials, which appear equally bright from all viewing directions. Finally, hypothesis (iii) is generally satisfied by the most cameras. Otherwise, the camera sensitivities can be sharpened by a linear transform [7, 19], in order to fulfill the requirements of (iii). Under these conditions, the von Kries model approximates the color change between the input images with a linear diagonal map, that rescales independently the color responses of the two pictures. Usually, the von Kries map is defined over the RGB color space, where the parameters of this model, called the von Kries coefficients, are the three scale factors that interrelate the red, green and blue responses of the input images. Despite its simplicity, the von Kries model has been proved to be a successful approximation of the illuminant change [17, 18] and it underpins many color constancy and color correction algorithms, e.g. [4, 9, 23, 29, 34, 35]. This success justifies the attention we dedicate to the von Kries model in this chapter, where we present three research studies, which have been dealt with in [35–37].

The work in [35] proposes a novel efficient method to estimate the parameters of the von Kries map possibly relating two images or two image regions. The estimation of the von Kries coefficients is carried out by the most popular color constancy algorithms, e.g. [11, 14, 20, 23, 33] in two steps: first, they compute the illuminants under which the input images or regions have been captured and codify them as 3D vectors; second, they estimate the von Kries coefficients as the ratios between the components of the illuminants. The approach in [35] strongly differs from these methods because it does not require any estimation or knowledge of the illuminants. It namely defines a von Kries model based measure of dissimilarity between the color distributions of the inputs, and derives the von Kries coefficients by minimizing this dissimilarity. The technique in [35] is invariant to many image distortions, like image rescaling, in-plane rotation, translation and skew, and it shows impressive performances in terms of computational charge, execution time, and accuracy on color correction, also in comparison with other methods, as shown from the experiments reported here.

The works in [36, 37] present some theoretical advances, that clarify the relationships of the von Kries parameters with the physical cues of the light (i.e. color temperature and intensity) and with the photometric properties of the device used for the image acquisition (i.e. the camera spectral sensitivities). To the best of our knowledge, the mathematical results discussed in these papers have been never proposed before.

The paper in [37] presents an empirical analysis of pictures or regions depicting the same scene or object under different Planck’s lights, i.e. lights behaving like a black-body radiator. The experiments carried out in [37] lead to two main results. First, a color change produced by varying two Planck’s lights is well approximated by a von Kries map. Such a result is not surprising, as it is in line with [18]. Second, the coefficients of the von Kries approximation are not independent to each other: they namely form 3D points belonging to a ruled surface, named von Kries surface, and parametrized by the color temperatures and intensities of the varied lights. The mathematical equation of the von Kries surface puts on evidence the relationship between the von Kries coefficients and the physical cues of the varied lights, and it is used in [37] to estimate the color temperature and intensity of an illuminant, given a set of images captured under that illuminants and a von Kries surface. Since in [37] there are no geometric distortions between the differently illuminated images (or regions) considered in the experiments, estimation of the von Kries approximation is performed by a best-fit technique, that computes the von Kries coefficients as the slope of the best line interpolating the pairs of pixel-wise correspondent color responses.

The work in [36] deals with changes of Wien’s illuminants, that are a special case of Planck’s lights. As in [37], and according to [18], an empirical study shows that the von Kries model is a valid approximation for the color changes due to a variation of Wien’s lights. Then the authors derive a mathematical equation interrelating the von Kries coefficients and the spectral sensitivities of the acquisition device. This equation reveals a direct proportionality between two 2D vectors: the first one is defined by the von Kries coefficients, while the second one by the wavelengths at which the RGB sensors of the device is maximally sensitive. This relationship allows to discover the relationship of the von Kries parameters with the photometric properties of the acquiring camera, and thus to prime information about the sensor sensitivities from the von Kries maps relating pairs of images linked by a variation of Wien’s illuminants. In the experiments reported in [36], estimation of the von Kries approximation was done by the approach of [35].

We notice that the hypotheses of Planck’s and Wien’s lights of the works [36, 37] do not compromise the generality of the results, because the most illuminants satisfy Planck’s or Wien’s law.

In the description of the works [35–37] provided in this chapter, we added more experiments and more comparisons with other approaches than those reported in the papers mentioned above. In particular, the method of [35] has been tested on three additional databases [8, 16, 45] and it has been compared with other techniques, listed in [13]. The experiments carried out in [37] have been repeated by estimating the von Kries approximation through the method [35], and a new measure for the accuracy on the von Kries approximation has been introduced (the Hilbert–Schmidt inner product of Sect. 5.2).

The overall organization of the Chapter is reported in Table 1.

Table 1 Chapter outline

2 Linear Color Changes

In the RGB color space, the response of a camera to the light reflected from a point $x$ in a scene is coded in a triplet $\mathbf{p}(x)$ = $(p_0(x)$, $p_1(x)$, $p_2(x))$, where

$$\begin{aligned} p_i (x) = \int _\Omega E(\lambda ) S(\lambda , x) F_i(\lambda ) \; d\lambda \qquad i = 0, 1, 2. \end{aligned}$$

(1)

In Eq. (1), $\lambda$ is the wavelength of the light illuminating the scene, $E$ its spectral power distribution, $S$ the reflectance distribution function of the illuminated surface containing $x$, and $F_i$ is the $i$-th spectral sensitivity function of the sensor. The integral ranges over the visible spectrum, i.e. $\Omega =4$ [380, 780] nm. The values of $p_0(x)$, $p_1(x)$, $p_2(x)$ are the red, green and blue color responses of the camera sensors at point $x$.

For a wide range of matte surfaces, which appear equally bright from all viewing directions, the reflectance distribution function is well approximated by the Lambertian photometric reflection model [44]. In this case, the surface reflectance can be expressed by a linear combination of three basis functions $S^k(\lambda )$ with weights $\sigma _k(x)$, $k$ = 0, 1, 2, so that Eq. (1) can be re-written as follows [42]:

$$\begin{aligned} \mathbf{p}(x)^T = W \mathbf{\sigma }(x)^T \end{aligned}$$

(2)

where $\mathbf{\sigma }(x) = (\sigma _0(x), \sigma _1(x), \sigma _2(x))$, the superscript $T$ indicates the transpose of the previous vector, and $W$ is the $3 \times 3$ matrix with entry

$$\begin{aligned} W_{ki} = \int _{\Omega } E(\lambda ) S^k(\lambda ) F_i(\lambda ) d\lambda , \qquad k, i = 0, 1, 2. \end{aligned}$$

The response $\mathbf{p'}(x) = (p'_0(x), p'_1(x), p'_2(x))$ captured under an illuminant with spectral power $E'$ is then given by $\mathbf{p'}(x)^T = W' \mathbf{\sigma }(x)^T$. Since the $\sigma (x)$’s do not depend on the illumination, the responses $\mathbf{p}(x)$ and $\mathbf{p'}(x)$ are related by the linear transform

$$\begin{aligned} \mathbf{p}(x)^T = W [W']^{-1} \mathbf{p'}(x)^T. \end{aligned}$$

(3)

Here we assume that $W'$ is not singular, so that Eq. (3) makes sense. In the following we indicate the $ij$-th element of $W [W']^{-1}$ by $\alpha _{ij}$.

3 The von Kries Model

The von Kries (or diagonal) model approximates the color change in Eq. (3) by a linear diagonal map, that rescales independently the color channels by real strictly positive factors, named von Kries coefficients.

Despite its simplicity, the von Kries model has been proved to approximate well a color changes due to an illuminant variation [15, 17, 18], especially for narrow-band sensors and for cameras with non-overlapping spectral sensitivities. Moreover, when the device does not satisfy these requirements, its spectral sensitivities can be sharpened by a linear transform [7, 19], so that the von Kries model still holds.

In the following, we derive the von Kries approximation for a narrow-band camera (Sect. 3.1) and for a device with non-overlapping spectral sensitivities (Sect. 3.2). In addition, we discuss a case in which the von Kries model can approximate also a color change due to a device changing (Sect. 3.3).

3.1 Narrow-Band Sensors

The spectral sensitivity functions of a narrow-band camera can be approximated by the Dirac delta, i.e. for each $i = 0, 1, 2, F_i(\lambda ) = f_i \delta (\lambda - \lambda _i)$, where $f_i$ is a strictly positive real number and $\lambda _i$ is the wavelength at which the sensor maximally responds.

Under this assumption, from Eq. (1), for each $i = 0, 1, 2$ we have

$$\begin{aligned} p_i(x) = E(\lambda _i)S(\lambda _i, x) F(\lambda _i) \qquad \mathrm{and} \qquad p_i'(x) = E'(\lambda _i)S(\lambda _i, x) F(\lambda _i) \end{aligned}$$

and thus

$$\begin{aligned} p_i(x) = \frac{E(\lambda _i)}{E'(\lambda _i)} p_i'(x) \qquad \forall \; i = 0, 1, 2. \end{aligned}$$

(4)

This means that the change of illuminant mapping $\mathbf{p}(x)$ onto $\mathbf{p'}(x)$ is a linear diagonal transform that rescales each channel independently. The von Kries coefficients are the rescaling factors $\alpha _{i}$, i.e. the non null elements $\alpha _{ii}$ of $W [W']^{-1}$:

$$\begin{aligned} \alpha _{i} := \alpha _{ii} = \frac{E(\lambda _i)}{E'(\lambda _i)} \qquad \forall \; i = 0, 1, 2. \end{aligned}$$

(5)

3.2 Non Overlapping Sensitivity Functions

Let $I$ and $I'$ be two pictures of a same scene imaged under different light conditions. Since the content of $I$ and $I'$ is the same, we assume a scene-independent illumination model [52] such that

$$\begin{aligned} E(\lambda ) F_k(\lambda ) = \sum _{j = 0}^2 \alpha _{kj} E'(\lambda )F_j(\lambda ). \end{aligned}$$

(6)

Now, let us suppose that the device used for the image acquisition has non overlapping sensitivity functions. This means that for each $i, j$ with $i \not = j$, $F_i(\lambda ) F_j(\lambda ) = 0$ for any $\lambda$. Generally, the spectral sensitivities are real-valued positive functions with a compact support in $\varOmega$ (see Fig. 1 for an example). Therefore non-overlapping sensitivities have non intersecting supports. We prove that under this assumption, the von Kries model still holds, i.e. matrix $W[W']^{-1}$ is diagonal.

Fig. 1

BARNARD2002: spectral sensitivities for the camera Sony DCX-930 used for the image acquisition of the database [8]

From Eq. (6) we have that

$$\begin{aligned} \int _\Omega E(\lambda ) S(\lambda , x) F_k(\lambda ) \; d\lambda = \sum _{j = 0}^2 \alpha _{kj} \int _\Omega E'(\lambda ) S(\lambda , x) F_j(\lambda ) \; d\lambda . \end{aligned}$$

(7)

i.e. the linear dependency between the responses of a camera under different illuminants is still described by Eq. (3). From Eq. (7) we have that

$$\begin{aligned}{}[E(\lambda ) F_k(\lambda ) - \sum _{j = 0}^2 \alpha _{kj} E'(\lambda ) F_j(\lambda ) ]^2 = 0. \end{aligned}$$

(8)

By minimizing (8) with respect to $\alpha _{kj}$ and by using the sensitivity non-overlap hypothesis we get the von Kries model. In fact, suppose that $k$ = 0. The derivative of the Eq. (8) with respect to $\alpha _{00}$ is

$$\begin{aligned} 0 = -E'(\lambda ) F_0(\lambda )[E(\lambda ) F_0(\lambda ) - \sum _{j=0}^{2} \alpha _{0j} E'(\lambda ) F_j(\lambda )]. \end{aligned}$$

Thanks to the non-overlapping hypothesis, and by supposing that $E'(\lambda ) \ne 0$ for each $\lambda$ in the support of $F_0$, we have that

$$\begin{aligned} E(\lambda ) F_0(\lambda ) - \alpha _{00} E'(\lambda ) F_0(\lambda ) = 0. \end{aligned}$$

(9)

By integrating Eq. (9) with respect to $\lambda$ over $\varOmega$, and by solving with respect to $\alpha _{00}$, we have that

$$\begin{aligned} \alpha _{00} = \frac{\int _\Omega E(\lambda )F_0(\lambda )d\lambda }{\int _\Omega E'(\lambda ) F_0(\lambda ) d\lambda }. \end{aligned}$$

(10)

Since $E$, $E'$ and $F_0$ are not identically null, $\alpha _{00}$ is well defined add $\alpha _{00} \ne 0$.

Now, we prove that $\alpha _{0j} = 0$ for any $j \ne 0$. From Eq. (9) we have that

$$\begin{aligned} E(\lambda ) F_0(\lambda ) = \alpha _{00} E'(\lambda ) F_0(\lambda ). \end{aligned}$$

Putting this expression of $E(\lambda ) F_0(\lambda )$ into Eq. (8) with $k$ = 0, yields

$$\begin{aligned} 0 = [\alpha _{01} E'(\lambda ) F_1(\lambda )]^2 + [\alpha _{02} E'(\lambda ) F_2(\lambda )]^2 + 2\alpha _{01}\alpha _{02} E'(\lambda )^2 F_1(\lambda ) F_2(\lambda ) . \end{aligned}$$

Since the functions $F_1$ and $F_2$ do not overlap, the last term at left is null, and

$$\begin{aligned}{}[\alpha _{01} E'(\lambda ) F_1(\lambda )]^2 + [\alpha _{02} E'(\lambda ) F_2(\lambda )]^2 = 0. \end{aligned}$$

By integrating this equation over $\lambda$ we have that

$$\begin{aligned} \alpha _{01}^2 \int _\Omega [E'(\lambda ) F_1(\lambda )]^2 d\lambda + \alpha _{02}^2 \int _\Omega [E'(\lambda ) F_2(\lambda )]^2 d\lambda = 0, \end{aligned}$$

(11)

and since $E'$, $E$, $F_0$, $F_1$, are not identically zero, we have that $\alpha _{01} = \alpha _{02} = 0$. By repeating the same procedure for $k = 1, 2$, we obtain the von Kries model.

We remark that Eq. (9) has been derived by supposing that $E'(\lambda )$ differs from zero for any $\lambda$ in the compact support of $F_0(\lambda )$. This allows us to remove the multiplicative term $E'(\lambda )F_0(\lambda )$ and leads us to Eq. (9). This hypothesis is reliable, because the spectral power distribution of the most illuminants is not null in the visible spectrum. However, in case of lights with null energy in some wavelengths of the support of $F_0$, Eq. (9) is replaced by

$$\begin{aligned} E'(\lambda ) E(\lambda )[ F_0(\lambda )]^2 - \alpha _{00} [E'(\lambda )]^2 [F_0(\lambda )]^2 = 0. \end{aligned}$$

The derivation of the von Kries model can be then carried out as before.

3.3 Changing Device

A color variation between two images of the same scene can be produced also by changing the acquisition device. Mathematically turns into changing the sensitivity function $F_i$ in Eq. (1). Here we discuss a case in which the color variation generated by a device change can be described by the von Kries model.

Without loss of generality, we can assume that the sensors are narrow bands. Otherwise, we can apply the sharpening proposed in [18] or [7]. Under this assumption, the sensitivities of the cameras used for acquiring the images under exam are approximated by the Dirac delta, i.e.

$$\begin{aligned} F_i(\lambda ) = f_i^{\gamma }\delta (\lambda - \lambda _i) \end{aligned}$$

(12)

where parameter $f^\gamma$ is a characteristic of the cameras.

Here we model the change of the camera as a variation of the parameter $\gamma$, while we suppose that the wavelength $\lambda _i$ remains the same. Therefore the sensitivity functions changes from Eq. (12) to the following Equation:

$$\begin{aligned} F'_i(\lambda ) = f_i^{\gamma ^*}\delta (\lambda - \lambda _i). \end{aligned}$$

(13)

Consequently, the camera responses are

$$\begin{aligned} p_i (x) = f_i^{\gamma } E(\lambda _i)S(\lambda _i, x)\qquad \mathrm{and}\qquad p_i'(x) = f_i^{\gamma ^*} E(\lambda _i)S(\lambda _i, x) \end{aligned}$$

and thus, therefore the diagonal linear model still holds, but in this case, the von Kries coefficients $\alpha _i$ depends not only on the spectral power distribution, but also on the device photometric cues:

$$\begin{aligned} \alpha _i = \frac{f_i^{\gamma } E(\lambda _i)}{f_i^{\gamma ^*} E(\lambda _i)}, \qquad \forall \; \; i = 0, 1, 2. \end{aligned}$$

(14)

4 Estimating the von Kries Map

The color correction of an image onto another consists into borrow the colors of the first image on the second one. When the color variation is caused by a change of illuminant, and the hypotheses of the von Kries model are satisfied, the color transform between the two pictures is determined by the von Kries map. This equalizes their colors, so that the first picture appears as it would be taken under the illuminant of the second one. Estimating the von Kries coefficients is thus an important task to achieve color correction between images different by illuminants.

The most methods performing color correction between re-illuminated images or regions compute the von Kries coefficients by estimating the illuminants $\sigma$ and $\sigma '$ under which the images to be corrected have been taken. These illuminants are expressed as RGB vectors, and the von Kries coefficients are determined as the ratios between the components of the varied illuminants. Therefore, estimating the von Kries map turns into estimating the image illuminants. Some examples of these techniques are the low-level statistical based methods as Gray-World and Scale-by-Max [11, 33, 55], the gamut approaches [4, 14, 20, 23, 29, 56], and the Bayesian or statistical methods [26, 41, 47, 50].

The method proposed in [35], we investigate here, differs from these techniques, because it does not require the computation of the illuminants $\sigma$ and $\sigma '$, but it estimates the von Kries coefficients by matching the color histograms of the input images or regions, as explained in Sect. 4.1. Histograms provide a good compact representation of the image colors and, after normalization, they guarantee invariance with respect to affine distortions, like changes of size and/or in-plane orientation.

As matter as fact, the method in [35] is not the only one that computes the von Kries map by matching histograms. The histogram comparison is in fact adopted also by the methods described in [9, 34], but their computational complexities are higher than that of the method in [35]. In particular, the work in [9] considers the logarithms of the RGB responses, so that a change in illumination turns into a shift of these logarithmic responses. In this framework, the von Kries map becomes a translation, whose parameters are derived from the convolution between the distributions of the logarithmic responses, with computation complexity $O(N\log (N))$, where $N$ is the color quantization of the histograms. The method in [34] derives the von Kries coefficients by a variational technique, that minimizes the Euclidean distance between the piecewise inverse of the cumulative color histograms of the input images or regions. This algorithm is linear with the quantizations $N$ and $M$ of the color histograms and of the piecewise inversions of the cumulative histograms respectively, so that its complexity is $O(N+M)$. Differently from the approach of [34], the algorithm in [35] requires the user just to set up the value of $N$ and its complexity is $O(N)$.

The method presented in [35] is described in detail in Sect. 4.1. Experiments on the accuracy and an analysis of the algorithm complexity and dependency on color quantization are addressed in Sects. 4.2 and  4.3 respectively. Finally, Sect. 4.4 illustrates an application of this method to illuminant invariant image retrieval.

4.1 Histogram-Based Estimation of the von Kries Map

As in [35], we assume that the illuminant varies uniformly over the pictures. We describe the color of an image $I$ by the distributions of the values of the three channels red, green, blue. Each distribution is represented by a histogram of $N$ bins, where $N$ ranges over {1, ..., 256}. Hence, the color feature of an image is represented by a triplet $\mathbf{H} := (H^0, H^1, H^2)$ of histograms. We refer to $\mathbf{H}$ as color histograms, whereas we name its components channel histograms.

Let $I_0$ and $I_1$ be two color images, with $I_1$ being a possibly rescaled, rotated, translated, skewed, and differently illuminated version of $I_0$. Let $\mathbf {H_0}$ and $\mathbf{H_1}$ denote the color histograms of $I_0$ and $I_1$ respectively. Let $H_0^i$ and $H_1^i$ be the $i$-th component of $\mathbf {H_0}$ and $\mathbf{H_1}$ respectively. Here fter, to ensure invariance to image rescaling, we assume that each channel $H_j^i$ histogram is normalized so that $\sum _{x = 1}^{N} H_j^i(x)$ = 1.

The channel histograms of two images which differ by illumination are stretched to each other by the von Kries model, so that

$$\begin{aligned} \sum _{k=1}^x H_1^i(k) = \sum _{k=1}^{\alpha _i x} H_0^i(k) \qquad \forall \;\; i = 0, 1, 2. \end{aligned}$$

(15)

We note that, as the data are discrete, the value $\alpha _i x$ is cast to an integer ranging over {1, ..., 256}.

The estimate of the parameters $\alpha _i$’s consists of two phases. First, for each $x$ in {1, ..., 256} we compute the point $y$ in {1, ..., 256} such that

$$\begin{aligned} \sum _{k = 1}^x H_0^i(k) = \sum _{k=1}^y H_1^i(k). \end{aligned}$$

(16)

Second, we compute the coefficient $\alpha _i$ as the slope of the best line fitting the pairs $(x, y)$.

The procedure to compute the correspondences $(x, y)$ satisfying Eq. (16) is implemented by the Algorithm 1 and more details are presented in [35]. To make the estimate robust to possible noise affecting the image and to color quantization, the contribution of each pair $(x, y)$ is weighted by a positive real number $M$, that is defined as function of the difference $\sum _{k = 1}^x H_0^i(k) - \sum _{k=1}^y H_1^i(k)$.

The estimate of the best line $\fancyscript{A}:= y = \alpha x$ could be adversely affected by the pixel saturation, that occurs when the incident light at a pixel causes the maximum response (256) of a color channel. To overcome this problem, and to make our estimate robust as much as possible to saturation noise, the pairs $(x, y)$ with $x =256$ or $y=256$ are discarded from the fitting procedure.

A least-square method is used to define the best line fitting the pairs $(x, y)$. More precisely, the value of $\alpha _i$ is estimated by minimizing with respect to $\alpha$ the following functional, that is called in [35] divergence:

$$\begin{aligned} d_{\alpha } (H_0^i, H_1^i) := \sum _k M_k d((x_k, y_k), \fancyscript{A})^2 = \sum _k \frac{M_k}{\alpha ^2 + 1}(\alpha x_k - y_k)^2. \end{aligned}$$

(17)

Here $(x_k, y_k)$ and $M_k$ indicate the $k$-th pair satisfying Eq. (16) and its weight respectively, while $d((x_k, y_k), \fancyscript{A})$ is the Euclidean distance between the point $(x_k, y_k)$ and the line $\fancyscript{A}$.

We observe that:

• 1. $d_{\alpha }(H_0^i, H_1^i)$ = 0 $\Leftrightarrow$ $H_0^i(\alpha z ) = H_1^i(z)$, for each $z$ in {1, ..., $N$};

• 2. $d_{\alpha }(H_0^i, H_1^i)$ = $d_{\frac{1}{\alpha }}(H_1^i, H_0^i)$.

These properties imply that $d_{\alpha }$ is a measure of dissimilarity (divergence) between the channel histograms stretched each to other. In particular, if $d_{\alpha }$ is zero, than the two histograms are stretched to each other.

Finally we notice that, when no changes of size or in-plane orientation occur, the diagonal map between two images $I_0$ and $I$ can be estimated by finding, for each color channel, the best line fitting the pairs of sensory responses $(p_i, p'_i)$ at the $i$-th pixels of $I_0$ and $I$ respectively, as proposed in [37]. The histogram-based approach in [35] basically applies a least square method in the space of the color histograms. Using histograms makes the estimate of the illuminant change insensitive to image distortions, like rescaling, translating, skewing, and/or rotating.

Figure 2 shows a synthetic example of pictures related by a von Kries map along with the color correction provided by the method described in [35]. The red channel of the image (a) has been rescaled by 0.5, while the other channels are unchanged. The re-illuminated image is shown in (b). Figure 3 shows the red histograms of (a) and (b) and highlights the correspondence between two bins. In particular, we note that the green regions in the two histograms have the same areas. The von Kries map estimated by [35] provides a very satisfactory color correction of image (b) onto image (a), as displayed in Fig. 2c.

Fig. 2

a A picture; b a re-illuminated version of (a); c the color correction of (b) onto (a) provided by the method [35]. Pictures (a) and (c) are highly similar

Fig. 3

Histograms of the responses of the red channels of the pictures shown in Fig. 2a, b: the red channel of the first picture has been synthetically rescaled by 0.5. The two red histograms are thus stretched to each other. The method [35] allows to estimate the stretching parameters, and hence to correct the images as they would be taken under the same light. The green parts highlighted on the histograms have the same area, therefore the bin $x = 128$ in the first histogram is mapped on the bin $y = 64$ of the second one

4.2 Accuracy on the Estimate

The accuracy on the estimate of the von Kries map possibly relating two images or two image regions has been measured in [35] in terms of color correction. In the following, we report the experiments carried out on four real-world public databases (ALOI [27], Outex [45], BARNARD [8], UEA Dataset [16]). Some examples of pictures from these datasets are shown in Fig. 4 (first and second images in each row).

Each database consists of a set of images (references) taken under a reference illuminant and a set of re-illuminated versions of them (test images). For all the databases, we evaluate the accuracy on the estimation of the the von Kries map $K$ by

$$\begin{aligned} A := 1 - L^1(I, K(I_0)). \end{aligned}$$

(18)

Fig. 4

Examples of color correction output by the approach in [35] for the four databases used in the experiments reported in Sect. 4.2: a ALOI, b Outex; c BARNARD; d UEA. In each row, from left to right: an image, a re-illuminated version of it, and the color correction of the second one onto the first one. The images in (d) have been captured by the same camera

Here $I$ indicates a test image and $I_0$ its correspondent reference, while $L^1(I, K(I_0))$ is the $L^1$ distance computed on the RGB space between $I$ and the color correction $K(I_0)$ of $I_0$ determined by the estimated $K$. This distance has been normalized to range over [0,1]. Therefore, the closer to 1 $A$ is, the better our estimate is. To quantify the benefit of our estimate, we compare the accuracy in Eq. (18) with

$$\begin{aligned} A_0 := 1 - L^1(I, I_0). \end{aligned}$$

(19)

The value of $A_0$ measures the similarity of the reference to the test image when no color enhancement is applied.

The transform $K$ gives the color correction of $I_0$ with respect to the reference $I$: in fact, $K(I_0)$ is the image $I_0$ as it would be taken under the same illuminant of $I$.

We notice that this performance evaluation does not consider possible geometric image changes, like rescaling, in-plane rotation, or skew. In fact, the similarity between two color corrected images is defined as a pixel-wise distance between the image colors.

In case of a pair of images related by an illuminant change and by geometric distortions, we measure the accuracy of the color correction by the $L^1$ distance between their color histograms. In particular, we compute the distance $\fancyscript{H}_0$ between the color histograms of $I$ and $I_0$ before the color correction

$$\begin{aligned} \fancyscript{H}_0 = 1 - L^1(\mathbf{H_0}, \mathbf{H}), \end{aligned}$$

(20)

and the distance $\fancyscript{H}$ between the color histograms $\mathbf{H}$ and $\mathbf{H_K}$ of $I$ and $K(I_0)$ respectively:

$$\begin{aligned} \fancyscript{H} = 1 - L^1(\mathbf{H}, \mathbf{H_K}). \end{aligned}$$

(21)

Examples of color correction output by the algorithm we described are shown in Fig. 4 for each database used here (third image in each row).

4.2.1 ALOI

ALOI [27] (http://staff.science.uva.nl/~aloi/) collects 110,250 images of 1,000 objects acquired under different conditions. For each object, the frontal view has been taken under 12 different light conditions, produced by varying the color temperature of 5 lamps illuminating the scene. The lamp voltage was controlled to be $V_j = j \times 0.047$ V with $j$ $\in$ {110, 120, 130, 140, 150, 160, 170, 180, 190, 230, 250}. For each pair of illuminants $(V_j, V_k)$ with $j \ne k$, we consider the images captured with lamp voltage $V_j$ as references and those captured with voltage $V_k$ as tests.

Figure 5 shows the obtained results: for each pair $(V_j, V_k)$, the plot shows the accuracies (a) $A_0$ and (b) $A$ averaged over the test images.

We observe that, for $j = 140$, the accuracy $A$ is lower than for the other lamp voltages. This is because the voltage $V_{140}$ determines a high increment of the light intensity and therefore a large number of saturated pixels, making the performances worse.

The mean value of $A_0$ averaged on all the pairs $(V_j, V_k)$ is 0.9913, while that of $A$ is 0.9961 by the approach in [35]. For each pair of images $(I_i, I_j)$ representing a same scene taken under the illuminants with voltages $V_i$ and $V_j$ respectively, we compute the parameters $(\alpha _0, \alpha _1, \alpha _2)$ of the illuminant change $K$ mapping $I_i$ onto $I_j$. In principle, these parameters should be equal to those of the map $K'$ relating another pair $(I'_i, I'_j)$ captured under the same pair of illuminants. In practice, since the von Kries model is only an approximation of the illuminant variation, the parameters of $K$ and $K'$ generally differ. In Fig. 6 we report the mean values of the von Kries coefficients versus the reference set. The error bar is the standard deviation of the estimates from their mean value.

Fig. 5

ALOI: accuracy a $A_0$ and b $A$ (see Eqs. (19) and (18)) for the different pairs of reference and test sets. The $x$ and $y$ axes display the lamp voltages (${\times }$0.047 V) of the illuminants used in ALOI. The right axis shows the correspondence between the colors of the plots and the values of the accuracies

4.2.2 Outex Dataset

The Outex database [45] (http://www.outex.oulu.fi/) includes different image sets for empirical evaluation of texture classification and segmentation algorithms. In this work we extract the test set named Outex_TC_00014: this consists of three sets of 1360 texture images viewed under the illuminants INCA, TL84 and HORIZON with color temperature 2856, 4100 and 2300 K respectively.

The accuracies $A_0$ and $A$ are stored in Table 2, where three changes of lights have been considered: from INCA to HORIZON, from INCA to TL84, from TL84 to HORIZON. As expected, $A$ is greater than $A_0$.

Fig. 6

ALOI: estimates of the von Kries coefficients

Table 2 Outex: accuracies $A_0$ and $A$ for three different illuminant changes

4.2.3 BARNARD

The real-world image dataset [8] (http://www.cs.sfu.ca/~colour/), that we refer as BARNARD, is composed by 321 pictures grouped in 30 categories. Each category contains a reference image taken under an incandescent light Sylvania 50MR16Q (reference illuminant) and a number (from 2 to 11) of relighted versions of it (test images) under different lights. The mean values of the accuracies $A_0$ and $A$ are shown in Fig. 7. On average, $A_0$ is 0.9447, and $A$ is 0.9805.

Fig. 7

BARNARD: accuracies $A_0$ (Eq. 19) and $A$ (Eq. 18) for the different illuminants

4.2.4 UEA Dataset

The UEA Dataset [16] (http://www.uea.ac.uk/cmp/research/) comprises 28 design patterns, each captured under 3 illuminants with four different cameras. The illuminants are indicated by Ill A (tungsten filament light, with color temperature 2865 K), Ill D65 (simulated daylight, with color temperature 6500 K), and Ill TL84 (fluorescent tube, with color temperature 4100 K). We notice that the images taken by different cameras differ not only for their colors, but also for size and orientation. In fact, different sensors have different resolution and orientation. In this case, the accuracy on the color correction cannot be measured by the Eqs. (18) and (19), but it is evaluated by the histogram distances defined in Eqs. (20) and (21).

The results are reported in Table 3. For each pair of cameras $(i, j)$ and for each illuminant pair $(\sigma , \sigma ')$ we compute the von Kries map relating every image acquired by $i$ under $\sigma$ and the correspondent image acquired by $j$ under $\sigma '$, and the accuracies $\fancyscript{H}_0$ and $\fancyscript{H}$ on the color correction of the first image onto the second one. On average, the $L^1$ distance between the color histograms before the color correction is 0.0043, while it is 0.0029 after the color correction.

Table 3 UEA Dataset: Accuracies $\fancyscript{H}_0$ and $\fancyscript{H}$ respectively (a) before and (b) after the color correction.

4.3 Algorithm Complexity and Accuracy versus Color Quantization

The approach presented in [35] has a linear complexity with respect to the number of image pixels, and to the color quantization $N$. Therefore, it is very efficient in terms of computational charge: the execution time requested to estimate the von Kries coefficients for a pair of images of size 150 $\times$ 200 is less than 40 ms on a standard Pentium4 CPU Intel® Core$^\mathrm{TM}$ i7-870 2.93 GHz, for $N = 256$.

The color quantization is a user input. The experiments carried out on both synthetic and real-world databases [35] show that the accuracy on the color correction decreases by decreasing the number of bins used to represent the color distributions. Figure 8 reports the mean value of the accuracy on color correction for the database ALOI for different values of $N$ and for the twelve illuminants of the database: the best performances are obtained for $N = 256$.

Fig. 8

ALOI: mean accuracy versus the illuminants for different color quantizations

4.4 An Application: Illuminant Invariant Image Retrieval

The illuminant invariant image recognition problem is stated as follows: let us consider a set of known images (references) and let $I$ be an unknown image (query). The problem consists into find the reference $I_0$ that displays the same (possibly re-illuminated) content of the query, i.e. the reference that visually is the most similar to the query.

The illuminant invariant recognition technique described in [35] relies on the estimate of the von Kries transform possibly linking a reference and a test image. Following [35], we compute the von Kries maps that borrow the colors of each reference onto those of the query, and we associate a dissimilarity score to each of these transforms. The solution $I_r$ is the image reference whose von Kries transform $K(I_0)$ has the minimum score from $I$.

The dissimilarity score is defined in terms of the divergence in Eq. (17) between the color histograms $\mathbf {H}$ and $\mathbf {H_r}$ of $I$ and $I_r$ respectively:

$$\begin{aligned} \delta = \sum _i d_{\alpha _i} (H^i, H_r^i). \end{aligned}$$

(22)

where $\alpha _i$ are the von Kries coefficients we estimate. The solution of the image recognition problem is given by the image $I_r$ of D such that the score defined by Eq. (22) is minimum.

As pointed out in [35], due to the dependency of the divergence in Eq. (17) on the von Kries coefficients, $\delta$ is not a metric because it does not satisfy the triangular inequality. Nevertheless, it is a query-sensitive dissimilarity measure, because it depends on the query [3].

In the following, we say that a query $I$ is correctly recognized if the reference image $I_r$ of D minimizing the score in Eq. (22) is a re-illuminated version of $I$.

We test the accuracy of this algorithm for the illuminant invariant image retrieval on two real-world databases ALOI and UEA, that we have still described in Sects. 4.2.1 and 4.2.4. We choose the first one, because it contains a large number of images under different illuminants, and the second one because it includes images acquired under different lights and by different devices. Moreofer, UEA database has been used in [13] to measure the performances of a novel illuminant- and device invariant image recognition algorithm, with which we compare our results.

We evaluate the recognition performances of our approach by the average match percentile (AMP) defined as the ratio (averaged over the test images)

$$\begin{aligned} \mathrm{AMP} = \frac{Q - rank}{Q-1}, \end{aligned}$$

(23)

where $Q$ is the number of reference images and $rank$ is the position of the correct object in the list of the models sorted by visual similarity.

In the experiments reported in the next subsections and carried out on ALOI [27] and UEA [16], we consider a color quantization with 256 bins. As well as in the case of color correction, a coarser quantization produces worse results [35] in image retrieval.

4.4.1 ALOI

Each set of images captured under an illuminant with voltage $V_j$ have been matched against each set of images captured under another illuminant with voltage $V_k$, $k \ne j$. The resulting AMP is shown in Fig. 9: in all the cases, the AMP is very high (0.9999 on average). The worst AMP has been obtained for the images taken under the lamps with voltage 140 $\times$ 0.047 V: in fact, these pictures contain a high percentage of saturated pixels and, as highlighted in Sect. 4.2.1, the accuracy on the von Kries map estimate is lower than in the other cases.

Fig. 9

ALOI: AMP index for the different illuminants. The $x$ and $y$ axes display the lamp voltages (${\times }$0.047 V) of the illuminants used in ALOI

4.4.2 UEA Dataset

We use UEA Dataset to measure how the accuracy of the illuminant invariant image retrieval algorithm changes across (1) illuminant and device variations; (2) illuminant variations; and (3) device variations. The results have been also compared with the techniques described in [13].

Results across Illuminant and Device Variation—For each pair $(c, \sigma )$ of camera $c$ and illuminant $\sigma$ we take the images captured by $c$ under $\sigma$ as references and we match them with the images captured by a camera $c^*$ under an illuminant $\sigma ^*$. We repeat the experiments by varying all the cameras and the illuminants. When $(c, \sigma )$ and $(c^*, \sigma ^*)$ coincide, we do not make any comparison, because in this case the references coincide with the tests. Table 4 reports the AMP averaged with respect the illuminants and the cameras.

Table 4 UEA dataset: AMP by varying the cameras and the illuminants by using the method in [35]

Results across Illuminant Variation—Table 5 shows the AMP obtained by comparing images taken by the same camera under different illuminants. In these experiments, we fix a pair $(c, \sigma )$, we take the images captured by $c$ under $\sigma$ as references and we match them with the images captured by $c$ under an illuminant $\sigma ^*$ different from $\sigma$.

Table 5 UEA dataset—AMP for the four cameras and for the three illuminants (taken as references)

Table 6 UEA dataset—AMP broken down by illuminant

In Table 6 these AMP’s are broken down by illuminants and compared with the AMP’s output by using the histogram equalization method (HE) and the Gray World color balancing (GW), both described in [13]. The same results are broken down by camera in Table 7.

Results across Device Variation—Table 8 shows the AMP’s for GW, HE, and for the method in [35] across a change of device: the images acquired by a camera $c$ under a fixed illuminant $\sigma$ are used as references, and matched with the images acquired by the other cameras under $\sigma$. The values reported in Table 8 are averaged over the reference illuminants. The results obtained by the approach in [35] are better than those achieved by GW and HE just for Camera 1 and Camera 2. For Camera 3 and Camera 4 HE performs better. On average, the results of our approach are very close to those output by GW, while HE outputs a higher AMP. More detailed results are reported in Table 9.

We note that when the illuminant changes, but the device is fixed, the method proposed in [35] outperforms GW and HE. On the contrary, HE provides better results than the approach in [35] and GW when the camera varies. This is because the von Kries model poorly describes a change of camera. In Sect. 3 we discussed a possible mathematical model for approximating a device variation, based on a change of the $\gamma$ factor. However, this approximation does not take into account a possible shift of the wavelength at which a sensor maximally responds, resulting in a too coarse approximation of the variation of the sensitivity functions. On the contrary, HE does not consider any model for approximating the illuminant change, but it defines a new image representation that is illuminant- and (in many cases) device-independent. The main idea of HE relies on the following observation, empirically proved in [13] across a wide range of illuminant and devices: while changing the illuminant and the recording device leads to significant color variations, the rank orderings of the color responses is in general preserved. The desired invariant image description is hence obtained by a histogram equalization method that takes into account the rank ordering constancy. However, the gap between the results provided by [35] and HE is small.

Table 7 UEA dataset—AMP broken down by camera

Table 8 UEA dataset—AMP across a change of device

Table 9 UEA dataset—AMP across a change of device and illuminant when the von Kries model is used

5 von Kries Model: Dependence on Light

In this Section, we analyze the dependence of the von Kries coefficients on the physical cues of the varied lights (i.e. color temperature and intensity). According to the work [37] result is obtained from an empirical analysis of pictures imaged under Planck’s lights, i.e. illuminants satisfying Planck’s law. As pointed out in Sect. 1, this assumption does not compromise the generality of the presented results, because many lights fulfill Planck’s law.

The empirical analysis proposed in [37] lead us to two main results: (i) we verify that the von Kries map well approximates a change of colors due to a change of Planck’s light; (ii) Planck’s law constraints the 3D points whose components are the von Kries coefficients, to lye on a ruled surface, called von Kries surface and parametrized by the physical properties of the light. The approximated equation of the von Kries surface, we derive following [37], reveals the relationship of the von Kries coefficients with the color temperature and intensity of the illuminants.

In Sect. 5.1, we describe Planck’s law and the Bradford transform, that is commonly used to model a color change due to a Planck’s illuminant variation. In Sect. 5.2 we discuss the result (i), where—differently from [37]—the von Kries approximations have been computed by the method in [35]. In Sect. 5.3 we illustrate the result (ii), and finally in Sect. 5.4 we explain how the von Kries surfaces can be used for estimating the color temperature and intensity illuminant of an image.

5.1 Planck’s Lights and Bradford Transform

A Planck’s illuminant is a light satisfying Planck’s law, i.e. its spectral power distribution is analytically expressed by the following formula:

$$\begin{aligned} E(\lambda , T, I) = J c_1 \lambda ^{-5} \Big (e^{\frac{c_2}{T \lambda }} - 1\Big )^{-1}. \end{aligned}$$

(24)

In Eq. (24), variables $\lambda$, $J$ and $T$ denote respectively the wavelength, the intensity and the color temperature of the illuminant. The terms $c_1$ and $c_2$ are constants ($c_1 = 3.74183 \cdot 10^{-16}$ W m$^2$ and $c_2 = 1.4388 \cdot 10^{-2}$ K m, with W $=$ Watt, m $=$ meter, K $=$ Degree Kelvin).

The intensity $J$ describes the illuminant brightness, while the color temperature measures the illuminant hue in Degrees Kelvin. For instance, the sun light at sunrise or at sunset has a color temperature between 2000 and 3000 K, while the color temperature of a candle flame ranges over [1850, 1930] K.

Usually, the color of a Planck’s light is codified as the 2D vector of its chromaticities in the CIE XYZ color space. The chromaticities of the most Planck’s illuminants have been tabulated empirically [30, 31], and approximated formulas are also available [43].

The Bradford transform [32, 57] is commonly used to model a color change due to a variation of two Planck’s illuminants $\sigma$ and $\sigma '$. This transform relates the XYZ coordinates $[X, Y, Z]$ and $[X', Y', Z']$ of the responses $\mathbf{p}$ and $\mathbf{p}^\prime$ by the linear map

$$\begin{aligned}{}[X', Y', Z']^T = M D M^{-1} [X, Y, Z]^T, \end{aligned}$$

(25)

where $M$ is the Bradford matrix and $D$ is a diagonal matrix encoding the relationship between the colorimetric properties (color temperatures and intensities) of $\sigma$ and $\sigma '$.

Bradford matrix has been obtained empirically from Lam’s experiments described in [32]:

$$\begin{aligned} M = \left[ \! {\begin{array}{ccc} 0.8951 &{} 0.2664 &{} -0.1614\\ -0.7502 &{} 1.7135 &{} 0.0367\\ 0.0389 &{} -0.0685 &{} 1.0296\\ \end{array} }\! \right] \; \mathrm{and } \; D = \frac{Y_\sigma }{Y_{\sigma ^\prime }} \left[ \! {\begin{array}{ccc} \frac{x_\sigma }{y_\sigma } \; \frac{y_{\sigma '}}{x_{\sigma '}} &{} 0 &{} 0 \\ 0 &{} 1 &{} 0\\ 0 &{} 0 &{} \frac{1 - x_\sigma - y_\sigma }{y_\sigma }\; \frac{y_{\sigma '}}{1 - x_{\sigma '} - y_{\sigma '}} \\ \end{array} } \!\right] \end{aligned}$$

where $[x_\sigma , y_\sigma ]$ and $[x_{\sigma '}, y_{\sigma '}]$ are the chromaticities of the color temperatures of $\sigma$ and $\sigma '$ respectively. $Y_\sigma$ and $Y_{\sigma '}$ are the $Y$ coordinates of the white reference of the illuminants $\sigma$ and $\sigma '$ respectively.

In the RGB space, the Bradford transform at an image pixel $x$ can be re-written as follows:

$$\begin{aligned} \mathbf{p'} ^T (x) = C M D M^{-1} C^{-1} \mathbf{p}^T(x):= B \mathbf{p}^T(x), \end{aligned}$$

(26)

where $C$ is the 3$\times$3 matrix mapping the XYZ coordinates into the RGB coordinates and $B := C M D M^{-1} C^{-1}$.

5.2 von Kries Approximation of a Bradford Transform

The illuminants of the images in the databases Outex and UEA are Planck’s lights. Therefore, the color variation relating correspondent pixels of re-illuminated images of these test sets can be expressed by the Eq. (26).

According to [17], the experiments reported in Sect. 4.2 on Outex and UEA Dataset showed that the von Kries model provides a good approximation of these illuminant changes, being the accuracy on the color correction very high. Therefore, the matrix $B$ in Eq. (26) and the diagonal matrix $K = \mathrm{diag}(\alpha _0, \alpha _1, \alpha _2)$ representing the von Kries approximation of the Bradford transform described by $B$ must be similar. We quantify the similarity between $K$ and $B$ on a synthetic dataset, as proposed in [37].

Our synthetic database is built up by re-illuminating the images of Outex by a set $\fancyscript{T}$ of Bradford transforms. More precisely, for each illuminant $\sigma$ of the database (i.e. $\sigma =$ INCA, TL84, HORIZON), we considered 36 Bradford transforms, each mapping $\sigma$ onto a Planck’s illuminant with color temperature $T_{\sigma '} = 2500 + t 500$ K (and $t = 0, \ldots , 8$) and intensity $J_{\sigma '} = (0.5 + i 0.5) J_\sigma$ (and $i = 0, \ldots , 3$). Since for the database Outex there are no information about the intensity $J_\sigma$ of the source illuminants $\sigma$, the Bradford transforms of $\fancyscript{T}$ simply rescales $J_\sigma$ by the parameters 0.5, 1.0, 1.5, 2.0.

We re-illuminated each image $I$ of Outex by the Bradford transforms of $\fancyscript{T}$, then for each re-lighted version $I'$ of $I$, we estimate the von Kries map approximating the correspondent Planck’s change of $\fancyscript{T}$, and finally we correct the colors of $I'$ onto the colors of $I$. Analogously to the results obtained in Sect. 4.2, the accuracy of the color correction on this synthetic database is very high: in fact, the mean values of $A_0$ and $A$ are 0.78423 and 0.97770 respectively.

These results imply that the matrix $B$ and $K$ are quite similar, i.e.

$$\begin{aligned} B : = C M D M^{-1} C^{-1} \simeq K. \end{aligned}$$

(27)

If $C M D M^{-1} C^{-1} = K$, then $K$ and $D$ represent the same endomorphism over the RGB color space with respect to two different bases of ${\mathbf {R}}^3$. However, since the von Kries model is just an approximation of a color variation, the equality $B = K$ is never satisfied.

In this Chapter, we measure the difference between $B$ and $K$ by the Hilbert–Schmidt inner product between matrices, defined as

$$\begin{aligned} \langle K, B \rangle := \sum _i \langle K e_i, B e_i \rangle , \end{aligned}$$

where $\{e_i\}_{i=0, 1, 2}$ is the standard canonical basis for $\mathbf {R}^3$. It is easy to verify that $\langle K, B \rangle$ is the trace of the matrix product $K \cdot B$. We normalize the Hilbert–Schmidt inner product as follows:

$$\begin{aligned} \langle K, B \rangle _{norm} = \frac{\langle K, B \rangle }{\left[ \sum _{i, j} K_{ij}^2 \sum _{ij} B_{ij}^2 \right] ^{0.5}} \end{aligned}$$

(28)

where $K_{ij}$ and $B_{ij}$ indicate the element $ij$ of $K$ and $B$ respectively. Thus $\langle K, B \rangle _{norm}$ ranges over [-1, 1]. The closer the module of the Hilbert–Schmidt distance (28) is to 1, more accurate the approximation represented by the diagonal matrix $K$ is.

For the synthetic database considered here, the Hilbert–Schmidt distance averaged on the number of Bradford transforms is 0.9587. This value shows that the matrices $B$ and $K$ are really close to each other.

In [37], the authors model the non-perfect equality of the matrices $K$ and $B$ by the Equation

$$\begin{aligned} K = H B \end{aligned}$$

(29)

where $H$ is a 3$\times$3 non singular matrix. As matter as fact, $H$ is really close to the identity matrix: in the synthetic experiments presented here, the mean values of the diagonal elements is 0.92, while that of the non diagonal entries is about 0.02. We also notice that in our synthetic test we do not control the pixel saturation phenomenon, that some Bradford transforms of $\fancyscript{T}$ may generate, especially when the gap of color temperature and the intensity scale factor are greater than 3000 K and 1.5 respectively.

5.3 von Kries Surface

According to [37], the experiments reported in Sect. 5.2 show that, for a fixed camera, the coefficients of a von Kries approximation of a change of Planck’s illuminants are not independent to each other, but they belong to a ruled surface, parametrized by the color temperature and intensity of the varied lights. This surface is named von Kries surface, and its mathematical equation puts on evidence the relationship between the von Kries coefficients and the photometric cues of the light (see Fig. 10). Moreover, in order to understand how the von Kries coefficients vary across the devices, we considered the pairs of images from UEA dataset captured by different cameras, but related by the same Planck’s illuminant variation. As in [37], we discovered that different devices produce different von Kries surfaces. Therefore the von Kries surfaces can be used to characterize a device.

Fig. 10

Synthetic database (from Outex): von Kries surface for the illuminant INCA. The color temperature ranges over [2500, 6500 K], the intensity has been rescaled by 0.5, 1.0, 1.5, 2.0. The plot shows that the von Kries coefficients are non independent to each other

Let us give some mathematical details to prime the mathematical equation of a von Kries surface.

Let $\sigma$ and $\sigma '$ be two Planck’s lights, having respectively intensities $J_\sigma$ and $J_{\sigma '}$, and color temperatures $T_\sigma$ and $T_{\sigma '}$. Let $\tau$ be the Planck illuminant change mapping $\sigma$ onto $\sigma '$ and let $\fancyscript{K}$ be its von Kries approximation. In particular, we have that $\tau$ maps $T_\sigma$ onto $T_{\sigma '}$ and $J_\sigma$ onto $J_{\sigma '}$.

The von Kries surface relative to $\sigma$ is defined by the following Equations:

$$\begin{aligned} \alpha _i = \alpha _i(T_{\sigma '}, J_{\sigma '}), \qquad i = 0, 1, 2 \end{aligned}$$

(30)

where the $\alpha _i$’s are the von Kries coefficients of $\fancyscript{K}$.

By combining Eqs. (26) and (29), we get $\alpha _i = \frac{Y_\sigma }{Y_{\sigma '}} \alpha _i^*(T_{\sigma '})$, for all $i = 0, 1, 2$, where $\alpha _0^*$, $\alpha _1^*$ and $\alpha _2^*$ define the von Kries approximation of the transform $\tau ^*$ that maps the color temperature $T_\sigma$ onto the color temperature $T_{\sigma '}$, while leaves unchanged the light intensity $J_\sigma$. By observing that $\frac{Y_\sigma }{Y_{\sigma '}} = \frac{J_\sigma }{J_{\sigma '}}$, we have that

$$\begin{aligned} \alpha _i = \frac{J_\sigma }{J_{\sigma '}} \alpha _i^*(T_{\sigma '}). \end{aligned}$$

(31)

From the Eq. (29), we have that

$$\begin{aligned} \alpha _i^*(T_{\sigma '}) = \sum _{j = 0}^{2} h_{ij} b^*_{ji}(T_{\sigma '}), \qquad i = 0, 1, 2 \end{aligned}$$

(32)

where $h_{ij}$ is the $ij$-th element of $H$ and $b^*_{ji}$ is the $ij$-th element of the matrix $B^*$ associated to the linear transform $\tau ^*$. Therefore we have that

$$\begin{aligned} \alpha _i (J_{\sigma '}, T_{\sigma '})= \frac{J_\sigma }{J_{\sigma '}} \sum _{j = 0}^{2} h_{ij} b^*_{ii}(T_{\sigma '}). \end{aligned}$$

(33)

Equation (33) makes evident the relationship between the von Kries coefficients and the photometric cues of the illuminants $\sigma$ and $\sigma '$. Equation (33) describes a ruled surface depending on the intensity and on the color temperature of $\sigma '$. Varying discretely the intensity and the color temperature of the source illuminant $\sigma$ produces a sheaves of such surfaces. Therefore, each illuminant $\sigma$ defines a von Kries surface $\fancyscript{S}$, that is completely determined by the Equation $\alpha ^*(T_\sigma ')$ = $(\alpha _0^*(T_\sigma '), \alpha _1^*(T_\sigma '), \alpha _2^*(T_\sigma '))$, as each point on $\fancyscript{S}$ is a rescaled version of a point onto $\alpha ^*(T_\sigma ')$. As a consequence, a set of Bradford transforms changing $\sigma$ to another Planck’s light with different color temperature suffice to estimate the von Kries surface relative to $\sigma$, while changes of intensity are not needed.

As output from our experiments, since the von Kries coefficients depend on the device, the von Kries surface relative to any illuminant $\sigma$ differs from device to device.

5.4 An Application: Estimating Color Temperature and Intensity from a von Kries Surface

Here we briefly discuss how the von Kries surfaces can be used to the estimate of the color temperature and the intensity of an illuminant. This estimation is a crucial task for many imaging applications, as for instance [46, 51].

Let $\sigma$ be a Planck’s illuminant with known color temperature $T_\sigma$ and intensity $J_\sigma$. Let $\fancyscript{S}$ the von Kries surface of a camera $c$ with respect to the source illuminant $\sigma$. Let $I$ and $I'$ two pictures depicting the same scene captured by $c$ under the illuminants $\sigma$ and $\sigma '$ respectively, where $\sigma '$ is supposed to be Planckian.

We determine the color temperature $T_{\sigma '}$ and the intensity $J_{\sigma '}$ of $\sigma '$ from the von Kries surface $\fancyscript{S}$ as follows. First, we estimate the von Kries coefficients $\beta _0, \beta _1, \beta _2$ of the von Kries map relating $I$ and $I'$. Then, we compute the triplet $(\alpha _0, \alpha _1, \alpha _2)$ on the von Kries map having the minimum Euclidean distance from $(\beta _0, \beta _1, \beta _2)$. Finally, we compute the color temperature and intensity correspondent to $(\alpha _0, \alpha _1, \alpha _2)$ as the actual color temperature and intensity of $\sigma '$.

Here we report the case study presented in [37], where the authors considered the image pairs $(I, I')$, with $I$ being an Outex picture imaged under TL84, and with $I'$ being the same picture captured under INCA or HORIZON. Figures 11a, b show the von Kries surface $\fancyscript{S}$ relative to TL84. The coefficients of the von Kries map transforming (a) TL84 onto INCA and (b) TL84 onto HORIZON respectively are displayed over the von Kries surface in green: these estimates determine a range of color temperatures and intensities. The color temperature of INCA we estimate ranges over [3000, 4000] K, but the 70 % about of the estimates are closer to 3000 K than to 4000 K. The variability range of the estimates of the color temperature of HORIZON is [2000, 4000] K, with the most part of the data (about the 90 %) in [2000, 3000 K]. Similarly, the estimated von Kries coefficients determine a variability range for the intensity, with the 99 % of the estimates between 1.0 and 1.25 for INCA and between 0.75 and 1.0 for HORIZON.

The accuracy on these estimates is close to the actual value of the color temperatures and intensities of INCA and HORIZON, but it could be further refined by considering a finer grid of color temperatures in the computation of the von Kries surface and by restricting the search for the triplet minimizing the Euclidean distance with the surface to the ranges found before. Nevertheless, in general, obtaining an accurate estimate of these photometric parameters is a hard problem [51], also when calibrated images are used [22].

Fig. 11

Outex: estimates of the color temperature and of the intensity for the illuminants a INCA and b HORIZON by using the von Kries surface with respect to TL84. Adapted from [37]

6 von Kries Model: Dependence on Device

In Sect. 5, we observed that von Kries surfaces can be used to characterize a sensor, but their equation does not reveal the relationships of the von Kries coefficients with the camera photometric cues. This dependence has been investigated in the work in [36], that we describe in this Section.

Wien’s law holds for Planck’s lights with color temperature varying over [2000, 10500] K, and their spectral power distribution is approximated by Wien’s law. Following the work in [36], we combine the Wien’s law with the von Kries model (Sect. 6.1), and we show that—as well as for Planck’s lights—the coefficients of the von Kries approximation of a change of Wien’s illuminants are not independent to each other (Sect. 6.2). Then we derive a mathematical equation linking the von Kries coefficients to the spectral sensitivities of the device. Since spectral sensitivities are often not reported in the technical manuals of the most cameras, and their estimation requires that the camera and an image calibration target are available, this equation proposes an alternative way to estimate these data. In fact, it allows to recover the sensor spectral cues directly from the parameters of a Wien’s illuminant variation, and thus to characterize a camera from a set of image pairs depicting the same scene and related by a light change (Sect. 6.3). This von Kries model based camera characterization applies to the work in [19], where an illuminant invariant image representation, which requires to known the camera spectral sensitivities is calculated (Sect. 6.4).

6.1 von Kries Model under Wien’s Law

Wien’s law is a special case of Planck’s law: it holds for Planck’s light whose color temperature ranges over [2000, 10500] K. In this interval, $e^{-\frac{c_2}{T \lambda }} \simeq (e^{\frac{c_2}{T \lambda }} - 1)^{-1}$, so that the spectral power distribution of a Wien’s light is

$$\begin{aligned} E(\lambda , T, I) = J c_1 \lambda ^{-5} e^{-\frac{c_2}{T \lambda }}. \end{aligned}$$

(34)

Variables $\lambda$, $J$ and $T$, and the constant terms $c_1$ and $c_2$ has the same meaning described in Sect. 5.1 for Eq. (24). Daylight, direct sunlight, candle lights and many fluorescent lights satisfy Wien’s law in Eq. (34).

Similarly to the case of Planck’s lights, any change of Wien’s illuminant can be described by the Bradford transform in Eq. (25), that is well approximated by the von Kries model. In Sect. 5.2, we show that the accuracy on color correction provided by the method in [35] for Planck’s illuminants is very high: the same holds for variations of Wien’s lights. Further tests can be found in [36].

The mathematical expression of the spectral power distribution of a Wien’s illuminant allows to recover an interesting relationship between the von Kries coefficients and the photometric properties of the camera used for the image acquisition. Here we describe the mathematical study presented in [36].

Without loss of generality, we assume that the camera is narrow-band. Therefore, by Eq. (34) we have that

$$\begin{aligned} p_i (x) = E(\lambda _i)S(\lambda _i, x) F(\lambda _i) = J c_1 \lambda _i^{-5} e^{-\frac{c_2}{T \lambda _i}} S(\lambda _i, x) F_i. \end{aligned}$$

Let us now consider the chromaticities at an image pixel $x$, i.e. the ratios $\frac{p_k(x)}{p_i(x)}$ for any $k \ne i$, $k, i$ = 0, 1, 2, and $p_i(x) \ne 0$. We have that

$$\begin{aligned} \frac{p_k(x)}{p_i(x)} = \Big (\frac{\lambda _i}{\lambda _k}\Big )^5 \frac{F_k S(\lambda _k, x)}{F_i S(\lambda _i, x)} e^{-\frac{c_2}{T} \Big (\frac{1}{\lambda _k} - \frac{1}{\lambda _i}\Big )} \end{aligned}$$

and thus for each $p_k(x) \ne 0$

$$\begin{aligned} \log \frac{p_k(x)}{p_i(x)} = \log \Big [\Big (\frac{\lambda _i}{\lambda _k}\Big )^5 \frac{F_k S(\lambda _k, x)}{F_i S(\lambda _i, x)}\Big ] - \frac{c_2}{T} \Big (\frac{1}{\lambda _k} - \frac{1}{\lambda _i}\Big ). \end{aligned}$$

(35)

By Eq. (35), the log-chromaticity $\log \frac{p_k(x)}{p_i(x)}$ is expressed as the sum of two terms: the first one depends just on the reflectance function and on some properties of the camera, while the second one depends just on the color temperatures $T$ and $T'$, and on the wavelength $\lambda _k$ and $\lambda _j$ at which the $k$th and the $i$th sensors maximally respond. If $\mathbf p '(x)$ is the response of the same camera at a pixel $x$ under a different Wien’s illuminant with color temperature $T'$, we have that

$$\begin{aligned} \log \frac{p_k(x)}{p_i(x)} - \log \frac{p_k'(x)}{p_i'(x)} = c_2\Big (\frac{1}{T} - \frac{1}{T'}\Big ) \Big (\frac{1}{\lambda _k} - \frac{1}{\lambda _i}\Big ). \end{aligned}$$

(36)

By the von Kries model, $p_k(x) = \alpha _k p'_k(x)$ ($k = 0, 1, 2$), and thus

$$\begin{aligned} \Big [ \log \frac{\alpha _k}{\alpha _i}, \log \frac{\alpha _j}{\alpha _i} \Big ] = c_2 \left( \frac{1}{T} - \frac{1}{T'}\right) \left[ \frac{1}{\lambda _k} - \frac{1}{\lambda _i}, \frac{1}{\lambda _j} - \frac{1}{\lambda _i}\right] . \end{aligned}$$

(37)

Equation (37) leads us to two main issues, that we discuss in the next Subsections.

6.2 von Kries Coefficients are Constrained by Wien’s Law

As for Planck’s lights, the coefficients of a von Kries map approximating a Wien’s illuminant change are not independent to each other. More precisely, from Eq. (37) we get the following Equations:

$$\begin{aligned} \left\{ \begin{array}{cc} \alpha _k = e ^{c_2\left( \frac{1}{T} - \frac{1}{T'}\right) \left( \frac{1}{\lambda _k} - \frac{1}{\lambda _i} \right) } \alpha _i \\ \alpha _j = e ^{c_2 \left( \frac{1}{T} - \frac{1}{T'}\right) \left( \frac{1}{\lambda _j} - \frac{1}{\lambda _i} \right) } \alpha _i \end{array} \right. \end{aligned}$$

(38)

Equation (38) show that the von Kries coefficients $\alpha _{k}$ and $\alpha _j$ are linearly proportional to $\alpha _i$ through a constant that depends on the camera photometric cues and on the color temperatures of the varied illuminants.

Moreover, we observe that when the source and target illuminants have the same color temperature, i.e. $T = T'$, then $\alpha _k = \alpha _i = \alpha _j$. Of course, the left term of Eq. (37) is zero. This implies that the illuminant variation is just an intensity variation, and the von Kries coefficients correspond to the scale factors between the intensities of the source and target lights.

The linear dependency between the von Kries coefficients expressed by Eq. (38) is generally not perfectly satisfied in the real-world use cases, i.e. the ratios $\frac{\alpha _k}{\alpha _i}$ and $\frac{\alpha _k}{\alpha _j}$ are not constant. This is due to two main reasons. First of all, the images are affected by noise, as for instance pixel saturation. Secondly, the Dirac delta is just a rough approximation of the spectral sensitivities of a camera.

6.3 Device Characterization Through von Kries Maps

Equation (37) states that the von Kries log-chromaticity vector $\underline{\alpha }$ and the camera sensitivity vector $\underline{\lambda }$

$$\begin{aligned} \underline{\alpha }:= \Big [ \log \frac{\alpha _k}{\alpha _i}, \log \frac{\alpha _j}{\alpha _i} \Big ] \qquad \underline{\lambda }:= \Big [ \frac{1}{\lambda _k} - \frac{1}{\lambda _i}, \frac{1}{\lambda _j} - \frac{1}{\lambda _i} \Big ] \end{aligned}$$

are parallel. Following [36], we observe that the versor

$$\begin{aligned} \underline{u} :=\frac{\underline{\alpha }}{\parallel \underline{\alpha }\parallel } \end{aligned}$$

(39)

and the vector $\underline{\lambda }$ defines the same line in $\mathbf R^2$, because

$$\begin{aligned} \underline{u} = \pm \frac{\underline{\lambda }}{\parallel \underline{\lambda }\parallel } \end{aligned}$$

(40)

Versor $\underline{u}$ does not depend on the physical properties of the light (i.e. intensity and color temperature), apart from its sign that is determined by the ratio

$$\begin{aligned} \frac{\frac{1}{T} - \frac{1}{T'}}{\Big | \frac{1}{T} - \frac{1}{T'}\Big |}. \end{aligned}$$

The versors $\underline{u}$ and $\frac{\underline{\lambda }}{\parallel \underline{\lambda }\parallel }$ can be expressed as

$$\begin{aligned} \underline{u} = (\cos \theta , \sin \theta ) \qquad \mathrm{and } \qquad \frac{\underline{\lambda }}{\parallel \underline{\lambda }\parallel } = (\cos \theta ^*, \sin \theta ^*) \end{aligned}$$

(41)

where $\theta$ and $\theta ^*$ are respectively the angle between $\underline{u}$ and the $x$-axis and the angle between $\underline{\lambda }$ and the $x$-axis.

From Eq. (40), we have that

$$\begin{aligned} \theta = \theta ^* + n\pi , \qquad n \in \mathbf Z, \end{aligned}$$

(42)

i.e. knowing the von Kries map that approximates a change of Wien’s illuminants allows to recover the parameter $\theta ^*$ that describes a photometric characteristic of the acquisition device.

The variations of Planck’s lights has been used in Sect. 5.3 to characterize a device through its von Kries surfaces. Here the approximation of the variations of Wien’s illuminants with the von Kries model allows to characterize the camera through an estimate $\theta$ of the parameters $\theta ^*$. Different devices can have different values of $\theta ^*$ and, according to this fact, for any fixed illuminant change $\tau$, the von Kries approximations of $\tau$ produced by these devices can differ to each other. However, we remark that the estimate in Eq. (42) differs from the actual value of $\theta ^*$ by $n\pi$, where $n$ is an integer number.

We observe that generally $\theta ^* \ne n\pi$ for any $n$ in $\mathbf Z$. In fact, if $\theta ^* = n\pi$, $\theta ^*$ should be parallel to the $x$-axis, and consequently we should have a camera with $\lambda _j = \lambda _k$, that in practice does not happen for RGB devices. Analogously, we exclude the case $\theta ^* = 0.5\pi + n\pi$, that implies $\lambda _k$ = $\lambda _i$.

As pointed out in Sect. 6.2, for intensity changes the vector $\underline{\alpha }$ is null and thus such light variations are inadequate to estimate $\theta ^*$. For the same reason, slight changes of color temperature are also inappropriate. A general recommendation to get a reliable estimate, is to consider illuminant changes where at least one of the components of $\underline{\alpha }$ is out of the range $(0.90, 1.10)$ [36].

In principle, the estimate of $\underline{u}$ can be done also when a single image $I$ and the color temperature $T$ of the illuminant under which it has been taken are known. Namely, in this case, we synthetically generate a second image $I'$ by re-illuminating $I$ with a Bradford transform that changes $T$ and then estimate the vector $\underline{\alpha }$. Nevertheless, due to the presence of many sources of noise like a large number of saturated pixels and/or high percentages of black pixels, it is recommendable to consider a set of images and then estimate $\underline{\alpha }$ by a statistical analysis of the data.

As matter as fact, since the Dirac delta are just a poor approximation of the spectral sensitivities of a camera, and since the images are often affected by noise (e.g. pixel saturation), the estimation of the angle $\theta ^*$ from the von Kries coefficients is generally intractable. What is feasible is to estimate a variability range of $\theta ^*$, as shown in the experiments on the databases Outex [45] and FUNT1998 [24], illustrated in the next Subsections.

6.3.1 Outex

Here we consider the dataset Outex [45]. In particular, we estimate $\theta ^*$ from the image pairs related by the following three illuminant changes: (a) from INCA to HORIZON; (b) from INCA to TL84; (c) from TL84 to HORIZON. The distributions of the estimates $\theta$ are displayed in Fig. 12: they show a peak close to their mean values, with a standard deviation equal to 0.22 (about $10^{\circ }$). The mean values of $\underline{u}$ are reported in Table 10.

Fig. 12

Real-world experiments: distributions of the direction $u$

6.3.2 FUNT1998

In this Section we describe the experiments carried out on the database [24] used in [36]. Here we refer to this database as FUNT1998.

Table 10 Outex: versors $\underline{u}$ for three illuminant changes

Table 11 FUNT1998: estimates of the von Kries coefficients for different illuminant pairs

Table 12 FUNT1998: estimates of the direction of $\underline{\lambda }$

The dataset FUNT1998 [24] contains 55 images of 11 objects taken under 5 different Wien’s illuminants and the spectral sensitivities of the camera used for image acquisition (a Sony DXC-930) are available and displayed in Fig. 1. By approximating each spectral sensitivity with a Dirac delta centered in the wavelength at which the sensitivity is maximum, the direction of $\underline{\lambda }$ is $\theta ^* = -0.9675$ rad. We compare this value with that output by the von Kries based estimation described before.

First of all, we consider the illuminant variation relating the image pairs taken under the following illuminant pairs: (halogen, syl-cwf), (halogen, mb-5000), (halogen, mb-5000+3202) and (halogen, ph-ulm). For each transform $\tau$ interrelating two illuminants, we compute the versor $\underline{u}$ from the von Kries approximation of $\tau$. The mean values of the von Kries coefficients (averaged on the number of image pairs) along with their standard deviations as error are displayed in Table 11.

On FUNT1998, the error on the estimate of the von Kries coefficients is quite high. Nevertheless, the accuracy on the color correction is good: when no color correction is performed, the mean accuracy $A_0$ is about 0.9682, while $A$ is 0.9884. Since the color change from halogen to ph-ulm is close to the identity map, it cannot be employed to estimate $\theta ^*$, because the log-chromaticities are null. Consequently, we just consider the illuminants syl-cwf, mb-5000 and mb-5000+3202. The estimates of $\theta ^*$ are listed in Table 12: the value of $\theta$ is the mean value of the estimates of $\underline{u}$ averaged over the 11 image pairs of FUNT1998, and the error is the standard deviation of the estimates from their mean value. The mean value of $\theta$ across the changes of illuminants is $-0.9655 \pm 0.1487$ rad.

6.4 An Application: Intrinsic Image Computation

In this Section, we explain how the estimate of $\theta ^*$ has been used in [36] to obtain the illuminant invariant image representation proposed in [19]. This intrinsic image is computed in [19] by observing that the log-chromaticity vector

$$\begin{aligned} \underline{\chi }(x) = \Big [\log \frac{p_k(x)}{p_i(x)}, \log \frac{p_j(x)}{p_i(x)} \Big ] \end{aligned}$$

(43)

belongs to a line with direction

$$\begin{aligned} \underline{\lambda } = \Big [ \frac{1}{\lambda _k} - \frac{1}{\lambda _i}, \frac{1}{\lambda _j} - \frac{1}{\lambda _i} \Big ]. \end{aligned}$$

(44)

Let $\underline{\lambda }^\perp$ be the 2D vector orthogonal to $\underline{\lambda }$. The intrinsic image of [19] is obtained by projecting the vector $\underline{\chi }(x)$ onto $\underline{\lambda }$, for each pixel $x$ of the input color picture. The intrinsic image (that is a one-channel image) is invariant to changes of Wien’s illuminants. In fact, for each $x$

$$\begin{aligned} \langle \underline{\chi }(x), \underline{\lambda }^\perp \rangle&= \log \Big [\Big (\frac{\lambda _i}{\lambda _k}\Big )^5 \frac{F_k S(\lambda _k, x)}{F_i S(\lambda _i, x)}\Big ] \Big (\frac{1}{\lambda _k} - \frac{1}{\lambda _i} \Big ) \\&\quad + \log \Big [\Big (\frac{\lambda _i}{\lambda _j}\Big )^5 \frac{F_j S(\lambda _j, x)}{F_i S(\lambda _i, x)}\Big ] \Big (\frac{1}{\lambda _j} - \frac{1}{\lambda _i} \Big ) \end{aligned}$$

and this value does not depend on the illuminant.

Unfortunately, the value of $\underline{\lambda }$ is generally unknown, because often the technical manuals of the most cameras do not report this information. When the camera or an image of a calibration target are available [12], the estimation of the camera sensitivity function is straightforward. However, in many applications, as for instance retrieving from the net pictures that are visually similar to an example, there are no information about the acquisition devices.

In order to overcome this lack, the work in [36] observes that the 2D versor $\underline{u}$ is parallel to $\underline{\lambda }$ and the scalar product $\langle \underline{\alpha }^\perp , \underline{\chi }(x) \rangle$ is also invariant to variation of Wien’s illuminants for each pixel $x$. To avoid possible ambiguities due to the sign of $\underline{u}$, the authors of [36] define a canonical orientation, for instance we require—as suggested above—that the basis $\{\underline{u}^\tau , (\underline{u}^\tau )^\perp \}$ for $\mathbf R ^2$ is positively oriented.

6.4.1 Quasi-Intrinsic Image: Tests

Versor $\underline{u}$ differs from $\underline{\lambda }$ for the module and for the sign. According to [36], we call quasi-intrinsic image the illuminant invariant image representation obtained by using $\underline{u}$ instead of $\underline{\lambda }$ to distinguish it from the intrinsic image of the work [19]. Thus, the quasi-intrinsic image differs from that proposed in [15] by a multiplicative factor. As the intrinsic image, quasi-intrinsic images of pictures related by a Wien’s illuminant change are equal. Here we report the empirical analysis carried out in [36] to show that two quasi-intrinsic images computed from two images of the same scene captured under different Wien’s lights are similar. Similarity between two quasi-intrinsic images is defined as the $L^1$ distance between their values.

Table 13 Outex: the first column reports three illuminant changes; for each of them, the second column reports the mean $L^1$ RGB distances $d_{RGB}$ between the re-illuminated images; the third, fourth, and fifth columns report the mean $L^1$ distances $d_{qi}$ between the quasi-intrinsic images, computed by using three different estimates of $\underline{u}^\perp$: (a) $\underline{u}^\perp = (0.5747, -0.8183)$, (b) $\underline{u}^\perp = (0.8019; -0.5974)$, (c) $\underline{u}^\perp = (0.6828; -0.7306)$

Table  13 reports the similarity measures for the database Outex. The values are averaged across the number of images and normalized to range over [0, 1]. Since the estimates of $\underline{u}$ of Table  10 are very close to each other, we obtained similar distances for all the cases.

7 Conclusions

This Chapter investigated the von Kries model, that is commonly used to approximate the color change occurring between two images depicting the same scene, but captured under different lights. Three main issues, recently published in [35–37], have been addressed: estimation of the parameters of the von Kries model, and two theoretical studies, that reveals the dependency of the von Kries coefficients on the physical properties of the light and on the photometric cues of the camera. These results have been discussed in details, and many applications have been proposed. The experiments reported in the Chapter have been carried out on synthetic and real-world databases, all freely available, in order to allow the reproducibility and the verification of the results, and further comparisons with other approaches.