A Geometric Model of Brightness Perception and Its Application to Color Images Correction

Abstract

Human perception involves many features like contours, shapes, textures, and colors to name a few. Whereas several geometric models for contours, shapes and textures perception have been proposed, the geometry of color perception has received very little attention, possibly due to the fact that our perception of colors is still not fully understood. Nonetheless, there exists a class of mathematical models, gathered under the name Retinex, which aim at modeling the color perception of an image, which are inspired by psychophysical/physiological knowledge about color perception, and which can geometrically be viewed as the averaging of perceptual distances between image pixels. Some of the Retinex models turn out to be associated with an efficient image processing technique for the correction of camera output images. The aim of this paper is to show that this image processing technique can be improved by including more properties of the human visual system. To that purpose, we first present a generalization of the perceptual distance between image pixels by considering the parallel transport map associated with a covariant derivative on a vector bundle, from which can be derived a new image processing model for color images correction. Then, we show that the family of covariant derivatives constructed in Batard and Sochen (J Math Imaging Vis 48(3):517–543 2014) can model some color appearance phenomena related to brightness perception. Finally, we conduct experiments in which we show that the image processing techniques induced by these covariant derivatives outperform the original approach.

1 Introduction

1.1 Geometry of Brightness Perception and Image Processing

Due to physical and technological limitations of the acquisition process of a real-world scene by a digital camera, the output image is a degraded version of the original scene. Processing the camera output image so that it reproduces the appearance of the original real-world scene is a very challenging task that requires the understanding of both the camera image processing pipeline and the way the human visual system (HVS) processes the light it captures. This problem has been addressed since digital imaging has come up in the early 1960s and is still a open problem, which is partly due to the complexity of the human visual system. We refer the reader to [7] for more details.

The Retinex theory of Land and McCann [19] was a seminal contribution in modeling the color appearance of a scene. Based on the well-known “Mondrian” experiments, the authors developed an algorithm to reach that goal.

Since then, many Retinex formulations have been proposed in order to improve the results of the original algorithm and extend it to more complex scenes (see, e.g., [6, 20, 33, 34] for more details about Retinex theory, Retinex formulations, and their connections with vision).

From a geometrical point of view, several Retinex formulations can be described as follows: The perceived color of an image at a pixel location is the averaging of the perceived difference between the given pixel value and other pixel values in the image domain, the perceived difference between two pixel values being determined by the integration of a perceived gradient along a path joining the two pixels. This interpretation suggests that the key point in order to determine the color appearance of a scene is to define accurately its perceived gradient. In several Retinex formulations, the perceived gradient is related to Weber’s law in psychophysics, which is a key observation for the generalization of the Retinex formulations we propose in this paper.

One of the properties of the HVS that plays a key role in the perception of the colors of a scene is its ability of adaptation to the lighting conditions, called color constancy, and which ensures that the perception of the colors of a scene remain relatively constant under a relighting (see, e.g., [13] for a review on color constancy). Georgiev [14] argues that this ability of the HVS could be related to an invariance property of the perceived gradient with respect to relighting. Then, equipping the set of lighting transformations with a Lie group structure G, this invariance property makes the perceived gradient behave as a covariant derivative, which makes a color image $u :\varOmega \subset {\mathbb {R}}^2 \longrightarrow {\mathbb {R}}^3$ be modeled as a section of a G-associated vector bundle of rank 3 over $\varOmega$. Note that there is another view in perception which states that color constancy does not derive from neighborhood interactions as it occurs in Retinex formulations, but from statistical correlations of the perceived scene [15].

Bertalmío et al. [6] proposed a kernel-based Retinex (KBR) formulation that has several advantages with respect to basic Retinex implementations based on paths: its computational complexity is much lower and it is less affected by artifacts, noise, or haloes. Moreover, they showed that KBR is closely related to a variational model for color image correction [5] that encodes some properties of the HVS. Indeed, by performing local contrast enhancement, which is a well-known property of the HVS, the model improves the visual quality of the image. Moreover, by coupling local contrast enhancement with visual adaptation, the model is able to perform, under some assumptions on the image, color constancy by removing the color cast of the image. In a more recent work, Palma-Amestoy et al. [23] added an extra property of the HVS to the variational model, i.e., a gamma compression, which is an alternative to Weber’s law to modeling brightness perception, and which is related to Steven’s law in psychophysics [29]. Adding a gamma compression to the model makes the local contrast enhancement be stronger in dark areas than in the bright ones, which is a desirable property when dealing with underexposed images.

Brightness perception has also been incorporated in a denoising model by Shen [27], who replaced the Euclidean gradient in the Rudin–Osher–Fatemi model [26] by the so-called Weber contrast that encodes Weber’s law. A consequence of Weber’s law is that color variations, and noise in particular, are more visible in dark areas than in the bright ones, which makes the model denoise in a greater extent the dark areas of the image.

Finally, we proposed in [1] a geometrical generalization of the Euclidean variational model introduced in [5] to vector bundles, by making use of the parallel transport map associated with a covariant derivative. We obtained a new image processing model whose behavior depends on the covariant derivative involved. Nonetheless, no connection between this new variational model and color perception has been established and this is actually one of the goals of this paper, whose contribution is detailed in the following section.

1.2 Contribution

The contribution of this paper is threefold: First, we show in Sect. 3 that the variational model we constructed in [1] is connected to the Retinex theory of color perception, which validates the model, at least theoretically, to performing image processing inspired by vision. Then, we show in Sects. 4, 5 that the class of optimal covariant derivatives constructed in Batard and Sochen [2] that are parametrized by Lie group representations, can encode some color appearance phenomena related to color brightness, which suggests that the HVS perceives color brightness in an optimal manner. Finally, we analyze in Sect. 6 the properties of the variational model constructed in [1] induced by these covariant derivatives and we show on experiments in Sect. 7 that it provides image processing techniques that outperform the perceptually inspired Euclidean model developed in [5].

More generally, this paper can be viewed as an investigation upon the capacity of Lie group representations and covariant derivatives to model color perception and to match psychophysical/physiological data as well as to generate image processing algorithms that improve the visual quality of camera output images.

In more detail, the contribution of this paper is the following.

1.2.1 Establishment of a Connection Between a Variational Model for Image Processing Based on Covariant Derivatives and the Retinex Theory

In Sect. 3.1, we propose a geometrical reinterpretation of the original Retinex formulation of Land and McCann [19] and the KBR formulation Bertalmío et al. [6], which exhibits the concept of perceptual gradient of an image.

In Sect. 3.2, we follow the approach of Georgiev [14] and consider covariant derivatives as a model for the perceptual gradient of an image, from which we generalize the original Retinex and KBR formulations. The proposed generalization makes the parallel transport map associated with a covariant derivative determine the perceptual difference between two pixel values. Then, we show that the mathematical properties of the parallel transport map match with some properties of the HVS. Indeed, the parallel transport map makes the perceptual difference between two pixel values depend on the colors between the pixel locations, and it is well known that the perceptual difference between two areas on an image depends on the image content. Moreover, a nonflat covariant derivative makes the corresponding parallel transport map depend on the curve joining two points, which suggests that the perceptual difference between two pixel values depends on the selected path followed by the eye joining the two pixel locations, as illustrated in Fig. 2. To the best of our knowledge, the connection between the curve dependency of the parallel transport map and human vision has never been proposed.

In [6], the authors established a connection between KBR and the contrast enhancement model introduced in [5]. In this paper, we follow this approach and establish a connection between the proposed generalization of KBR approach and the contrast enhancement model we introduced in [1]. This point is developed in Sect. 3.3.

1.2.2 Analysis of a Class of Covariant Derivatives Encoding Properties of the HVS

Georgiev [14] encoded pixel transformations under relighting with the Lie group $D^{+*}_3$ of $3 \times 3$ strictly positive diagonal matrices, and constructed a covariant derivative that satisfies an invariance property with respect to the action of this group, which turns out to be related to Weber’s law. In this paper, we extend this approach in the sense that we consider more Lie group representations acting on a three-dimensional space and by considering an optimal covariant derivative associated with each group representation. Indeed, besides the group $D^{+*}_3$ aforementioned, we consider group representations of SO(2) and SO(3) on ${\mathbb {R}}^3$, and the corresponding optimal covariant derivatives are the ones constructed in Batard and Sochen [2] and that are image dependent. This point is developed in Sects. 4 and 5.

The main geometric object we study throughout this paper is the curvature of a covariant derivative, as the former provides fundamental information about the latter. For instance, the connection 1-form of a flat covariant derivative, i.e., a covariant derivative whose curvature is zero, vanishes with respect to some moving frame, from which follows that the covariant derivative corresponds to the standard differential operator when expressed in this moving frame. By studying the curvature of the optimal covariant derivatives constructed in [2], the main results we obtain are the following.

The standard representation of the group $D_3^{+*}$ on ${\mathbb {R}}^3$. The optimal covariant derivative induced by this representation is parametrized by $\alpha =(\alpha _1,\alpha _2,\alpha _3) \in {({\mathbb {R}}^{+*})}^3$, and we show that it is flat for any value of $\alpha$. Then, assuming that the space of the representation is a trichromatic color space (e.g., LMS or RGB), the group representation encodes pixel transformation under relighting as aforementioned.

At the limit value $(\alpha _1,\alpha _2,\alpha _3)=(0,0,0)$, we show that the optimal connection 1-form encodes Weber’s law, as in the original Retinex and KBR formulations.

We show that there exists a value of the parameter $\alpha$ for which the connection 1-form associated with the optimal covariant derivative encodes Weber–Fechner’s law in vision, which is a refinement of Weber’s law. As a consequence, the corresponding Retinex formulation shall provide a more accurate expression for the color perception of an image than the Retinex formulations aforementioned as these latter encode (the less accurate) Weber’s law. Moreover, assuming that the observed scene has a limited dynamic range (which is the case when observing an image on a standard display device), the three types of cones in the retina have response that follows Weber–Fechner’s law (see, e.g., [12] and references therein for more details), which makes this connection 1-form be physiologically plausible as well.

At the limit value $(\alpha _1,\alpha _2,\alpha _3)=(+\infty ,+\infty ,+\infty )$, the optimal connection 1-form vanishes, meaning that the optimal covariant derivative is nothing but the standard derivative.

Representations of SO(2) on ${\mathbb {R}}^3$. The representations of SO(2) on ${\mathbb {R}}^3$ are parametrized by the set of planes in ${\mathbb {R}}^3$, and a representation contains all the rotations that leave a given plane invariant. For a given representation, the optimal covariant derivative is parametrized by $\alpha \in {\mathbb {R}}^{+*}$. We show that the optimal covariant derivative is flat for any image only at the limit values $\alpha \in \{ 0,+\infty \}$. Then, assuming that the space of the representation is a trichromatic color space equipped with the Euclidean scalar product, we show that there exists a representation of SO(2) that encodes hue changes, which implies that the representation implicitly encodes the color space change from a trichromatic space to an opponent space of the form intensity-chrominance.

For the limit value $\alpha =0$, we show that the moving frame in which the connection 1-form vanishes splits the intensity and chrominance information, and exhibits the hue field introduced by Ben-Shahar et al. [4], which is physiologically implemented in the visual cortex.

Assuming that $\alpha \in {\mathbb {R}}^{+*}$, we show the existence of conditions on the image for which the covariant derivative is flat. More precisely, we show that the covariant derivative is flat if all the pixel values belong to a circle centered at the origin in the plane left invariant under the group representation. We deduce an explicit expression of the parallel transport map along a curve joining two pixels if the pixel values belong to such a circle along the curve. This result provides us more insight into the behavior of the covariant derivative.

Finally, at the limit value $\alpha =+\infty$, the connection 1-form vanishes.

The standard representation of SO(3) on ${\mathbb {R}}^3$. The optimal covariant derivative is parametrized by $\alpha \in {\mathbb {R}}^{+*}$, and we show that the covariant derivative is flat for any image only for the limit values $\alpha \in \{ 0,+\infty \}$.

Assuming that $\alpha =0$ and the space of the representation is the CIE $L^{*}a^{*}b^{*}$ color space, we show that, for a well-chosen vector bundle metric, the covariant derivative encodes the differential of the “color brightness,” defined by Fairchild and Pirrotta [11], this latter being a refinement of the standard brightness $L^{*}$, as it takes into account the visual effect called the Helmholtz–Kohlrausch (H–K) effect, which states that the brightness of a color object depends on both its standard brightness $L^{*}$ and its chrominance information $a^{*},b^{*}$.

Assuming that $\alpha \in {\mathbb {R}}^{+*}$, we show the existence of conditions on the image for which the covariant derivative is flat: It is flat if all the pixel values belong to a circle centered in 0, i.e., to a geodesic of the sphere $S^2$. We deduce an explicit expression of the parallel transport map along a curve joining two pixels if the pixel values belong to some geodesic of $S^2$ along the curve. As in the SO(2) case, this result provides us more insight into the behavior of the covariant derivative.

As for the previous groups studied, the connection 1-form vanishes at the limit value $\alpha =+\infty$.

The standard representation of $SO(3) \times D_3^{+*}$ on ${\mathbb {R}}^3$. Based on the flatness properties of the covariant derivatives induced by the groups $D_3^{+*}$ and SO(3) aforementioned, we construct a flat covariant derivative associated with the standard representation of SO(3) $\times D_3^{+*}$ on ${\mathbb {R}}^3$. More precisely, denoting by ${\mathcal {G}}_1$ the moving frame in which the optimal connection 1-form associated with $D_3^{+*}$ for $\alpha \in {\mathbb {R}}^{+*}$ vanishes, and by ${\mathcal {G}}_2$ the moving frame in which the optimal connection 1-form associated with SO(3) for $\alpha =0$ vanishes, we consider the covariant derivative whose connection 1-form vanishes in the moving frame ${\mathcal {G}}_2 {\mathcal {G}}_1$. This construction makes the covariant derivative be flat, but it does not guarantee that it is optimal, i.e., is a solution of a variational problem involving the standard representation of SO(3) $\times D_3^{+*}$ on ${\mathbb {R}}^3$.

1.2.3 Analysis and Implementation of a Class of Variational Models Performing Color Images Correction

In Sects. 6, 7, we consider the variational model constructed in [1] endowed with the different covariant derivatives analyzed in Sects. 4, 5. The study of the flatness property of the covariant derivatives and the construction of the moving frames in which the connection 1-forms of the flat ones vanish enables to get an insight into the behavior of the model. Indeed, the variational model induced by a flat covariant derivative can be written, up to a moving frame change, as the vectorial extension of the channel-wise model of Bertalmío et al. [5] that we proposed in [3]. This allows a much faster implementation of the algorithm as there is no need to compute the parallel transport map.

More precisely, we obtain the following results.

The standard representation of the group $D_3^{+*}$ on ${\mathbb {R}}^3$. We show that the model induced by the optimal covariant derivative yields a local contrast enhancement model that enhances more the dark than the bright areas of the image unless we consider the limit value $\alpha =(+\infty ,+\infty ,+\infty )$ that makes the model enhance both areas in a similar extent. This result is coherent with the results in Sects. 4 and 5, where we show that the connection 1-form encodes Weber’s law for $\alpha =(0,0,0)$ and Weber–Fechner’s law for some $\alpha \in ({{\mathbb {R}}^{+*}})^3$. This is a desirable property when dealing with underexposed images. Moreover, we show that the model provides better results when it encodes Weber–Fechner’s law than when it encodes Weber’s law.

Representations of SO(2) on ${\mathbb {R}}^3$. Assuming that $\alpha =0$, we show that the variational model consists in enhancing the local contrast of the intensity and chroma components of the image while preserving its hue.

The analysis of the behavior of the variational model in the case $\alpha \in {\mathbb {R}}^{+*}$ is less trivial as this value makes the covariant derivative be not flat. However, we show that the value $\alpha$ can serve as a parameter of regularization of the hue variations, which can be useful in some situations as the hue is very noisy at low intensity values for instance.

For $\alpha =\infty$, the model is nothing but the Euclidean model in [3].

The standard representation of SO(3) on ${\mathbb {R}}^3$. Assuming that $\alpha =0$ and the vector bundle is equipped with a well-chosen metric, the variational model induced by the optimal covariant derivative consists in enhancing the local contrast of the color brightness (according to the definition of Fairchild and Pirrotta [11]) of the original image, and we show that the model satisfies two properties. First, it preserves the hue of the original image, which is a desirable property when enhancing the contrast of some images (see, e.g., [22, 24] for other hue preserving contrast enhancement models). We show on experiments that this model creates less color artifacts than the model [5].

As in the case of the group SO(2) aforementioned, the fact that the covariant derivative is not flat for $\alpha \in {\mathbb {R}}^{+*}$ makes the behavior of the model much more difficult to analyze. However, we show that the value $\alpha$ can serve as a parameter of regularization of the hue variations and in a more accurate way than in the SO(2) case.

Finally, for $\alpha =+\infty$, we obtain the Euclidean model in [3].

The standard representation of $SO(3) \times D_3^{+*}$ on ${\mathbb {R}}^3$.

The covariant derivative aforementioned associated with the group representation of SO(3) $\times D_3^{+*}$ on ${\mathbb {R}}^3$ has been constructed in order for the subsequent variational model to combine the properties of the variational models induced by the optimal covariant derivatives associated with the group $D_3^{+*}$ for a well-chosen $\alpha \in D_3^{+*}$ and SO(3) for $\alpha =0$. Experiments show that the model does combine theses properties: It enhances more the local contrast of the image in the dark areas than in the bright ones, and it preserves the hue of the original image. Moreover, it preserves the saturation, defined as the ratio of chroma to intensity, of the original image. Comparing the three models on underexposed images shows that the best results are obtained with the model combining the two optimal covariant derivatives.

2 Definitions and Notations

2.1 On the Curvature Tensor and the Parallel Transport Map

One of the main geometric objects we consider in this paper is the curvature tensor. In this section, we recall its definition and its relation with the parallel transport map on a vector bundle. We refer the reader to [17] and ([2], Appendix A) for more details about the geometry of fiber bundles.

Let G be a Lie group and ${\mathfrak {g}}$ its Lie algebra, P be a principal G-bundle over a manifold M and $\pi _P :P \longrightarrow M$ the projection map. We denote by $P_x \simeq G$ the fiber over $x \in M$, i.e., the set $\{ \pi _P^{-1}(x) \}$, and by $\varGamma (P)$ the set of smooth sections of P. In what follows, we denote by $(P,\pi _P,M,G)$ the principal G-bundle P.

Let $\rho :G \longrightarrow GL({\mathbb {R}}^n)$ be a group representation of G on ${\mathbb {R}}^n$. We denote by $E_{\rho }$ the G-associated bundle $P \times _{\rho } {\mathbb {R}}^n$, and by $P(E_{\rho })=P \times _{\rho } End({\mathbb {R}}^n)$ the bundle of moving frames of $E_{\rho }$.

Given a manifold X, we denote by TX its tangent bundle, $T^{*}X$ its cotangent bundle, and by $\bigwedge ^2 T^{*}X$ the bundle of differential 2-forms of X.

Definition 1

(Curvature of a connection on a principal bundle) Let $\omega \in \varGamma (T^{*} P \otimes {\mathfrak {g}})$ be a connection 1-form on a principal bundle $(P,\pi _P,M,G)$. The curvature 2-form $F(\omega ) \in \varGamma (\bigwedge ^2 T^{*} P \otimes {\mathfrak {g}})$ of $\omega$ is defined by

$$\begin{aligned} F(\omega ) := d\omega + \dfrac{1}{2} [\omega ,\omega ]\end{aligned}$$

(1)

where d stands for the exterior derivative and $[\cdot , \cdot ]$ the wedge product of Lie algebra-valued forms.

The connection 1-form $\omega$ on P induces a connection 1-form $\omega _{\rho } \in \varGamma (T^{*} P \otimes End({\mathbb {R}}^n))$ on $P(E_{\rho })$ given by

$$\begin{aligned} \omega _{\rho }= \rho _{*} (-\omega ) \end{aligned}$$

and the curvature 2-form F(w) on P induces a curvature 2-form $F(\omega _{\rho }) \in \varGamma (\bigwedge ^2 T^{*} P \otimes End({\mathbb {R}}^n))$ on $P(E_{\rho })$ given by

$$\begin{aligned} F(\omega _{\rho })= \rho _{*} (-F(\omega )) \end{aligned}$$

where $\rho _{*} :{\mathfrak {g}} \longrightarrow End({\mathbb {R}}^n)$ is the Lie algebra representation associated with $\rho$.

Definition 2

(Flat connection) The connection 1-form $\omega$ is called flat if $F(\omega ) \equiv 0$.

We say that a covariant derivative is flat if the corresponding connection 1-form is flat.

Unlike the connection 1-forms, the curvature 2-forms are transformed in a tensorial way with respect to the action of G on the fibers of P. Indeed, given $s \in \varGamma (P)$, $g \in C^{\infty }(\varOmega ,G)$ and denoting by $\cdot$ the action of G on the fibers of P, we have

$$\begin{aligned} F_{|g \cdot s}= g^{-1} F_{|s}g. \end{aligned}$$

Hence, in order to show that a connection is flat, it is sufficient to show that it vanishes along a section of P.

Given $s \in \varGamma (P)$, the connection 1-form $\omega$ can be pull down on the base manifold M as a 1-form $s^{*} \omega \in \varGamma (T^{*}M \otimes {\mathfrak {g}})$, also called a connection 1-form, and it transforms in the following manner under the action of G

$$\begin{aligned} (g \cdot s)^{*} \omega = g^{-1} dg + g^{-1} (s^{*} \omega ) g \end{aligned}$$

for $g \in C^{\infty }(\varOmega ,G)$. The connection one-form $s^{*}\omega$ determines a connection one-form $A_{\rho } := \rho _{*}(-s^{*}\omega ) \in \varGamma (T^{*} M \otimes End({\mathbb {R}}^n))$ that transforms in the following manner under the action of G on P

$$\begin{aligned} \rho _{*}( -(g\cdot s)^{*}\omega ) = {\mathcal {G}}^{-1} d{\mathcal {G}} + {\mathcal {G}}^{-1}\rho _{*}(-s^{*}\omega ) {\mathcal {G}} \end{aligned}$$

(2)

where ${\mathcal {G}} \in \varGamma (P(E_{\rho }))$ is the moving frame of $E_{\rho }$ given by ${\mathcal {G}}= \rho (g)$.

From the pulldown of $\omega$ on the base manifold M through the section s, we can define a curvature 2-form $F(A) \in \varGamma (\bigwedge ^2 T^{*} M \otimes {\mathfrak {g}})$ on M, as

$$\begin{aligned} F(A) := dA+ \dfrac{1}{2} [A,A ]\end{aligned}$$

(3)

and the corresponding curvature 2-form

$F(A_{\rho }) \in \varGamma (\bigwedge ^2 T^{*} M \otimes End({\mathbb {R}}^n))$ is given by

$$\begin{aligned} F(A_{\rho })=\rho _{*}(-F(A)). \end{aligned}$$

(4)

Given a covariant derivative $\nabla =d+\omega _{\rho }$ on the vector bundle $E_{\rho }$, the corresponding parallel transport map is defined as follows.

Definition 3

(Parallel transport) Let $\gamma :I \subset {\mathbb {R}} \longrightarrow M$ be a smooth curve. The parallel transport associated with $\nabla$ along the curve $\gamma$ is the map $\tau _{t_1,t_2,\gamma } :E_{\gamma (t_1)} \longrightarrow E_{\gamma (t_2)}$ such that $\tau _{t_1,t_2,\gamma } (u_0)$ is the solution of the differential equation

$$\begin{aligned} \left\{ \begin{array}{ccl} \nabla _{_{ \gamma '(t)}} \, u (\gamma (t)) &{} = &{} 0 \quad \forall t \in [t_1,t_2 ]\\ \quad u (\gamma (t_1)) &{} = &{} u_0 \end{array} \right. \end{aligned}$$

(5)

Finally, it follows from the results aforementioned that the parallel transport map associated with a flat connection writes as the Identity map in the moving frame ${\mathcal {G}} \in P(E_{\rho })$ in which it vanishes, and consequently does not depend on the curve (in a homotopy class) joining the points.

2.2 Nonlocal Total Variation on a Vector Bundle and Its Dual Formulation

The construction of the solutions of the variational model we consider in this paper and their numerical implementation requires the use of several notions that we introduced in [1] and that we recall in this section.

Let E be a vector bundle of rank n over a manifold M equipped with a positive definite metric h. We denote by $\pi _E :E \longrightarrow M$ the projection map, by $E_x \simeq {\mathbb {R}}^n$ the fiber over $x \in M$, i.e., the set $\{ \pi _E^{-1}(x) \}$, and by $\varGamma (E)$ the set of smooth sections of E. We denote by $pr_1(E)$ the vector bundle over $M \times M$ induced by the projection

$$\begin{aligned} \begin{array}{cc} &{} M \times M \longrightarrow M \\ pr_1 : &{} (x,y) \longmapsto x \end{array} \end{aligned}$$

In other words

$$\begin{aligned} pr_1(E)=\{ (x,y,p) \in M \times M \times E /\, x=\pi _E(p) \}. \end{aligned}$$

(6)

Definition 4

(Nonlocal covariant derivative) The nonlocal covariant derivative associated with a covariant derivative $\nabla =d+\omega$ on E is an operator

$\nabla ^{NL}_w :\varGamma (E) \longrightarrow \varGamma (pr_1 (E))$ of the form

$$\begin{aligned} \nabla ^{NL}_{w} \, u \, :(x,y) \longmapsto w(x,y)\left( \tau _{0,T,\gamma _{y,x}} \,u(y)-u(x) \right) \end{aligned}$$

(7)

where $\gamma _{y,x}$ is a curve joining y and x of length T, and $w :M \times M \longrightarrow {\mathbb {R}}^{+*}$ is a smooth symmetric function.

The positive definite metric h on E induces a $L^2$ scalar product $\langle \, \, , \, \, \rangle$ on $\varGamma (pr_1(E))$ defined by

$$\begin{aligned} \langle \eta _1,\eta _2 \rangle := \int _{M \times M} (\eta _1(x,y),\eta _2(x,y))_{h(x)} \, \mathrm{d}x \, \mathrm{d}y \end{aligned}$$

where $( \, \,, \, )_{h(x)}$ denotes the scalar product in $E_x$ with respect to h, from which derive the $L^p$ norm on $\varGamma (pr_1(E))$ defined by

$$\begin{aligned} \Vert \eta \Vert _{L^p} := \left( \int _{M \times M} \Vert \eta (x,y) \Vert _{h(x)}^p \, \mathrm{d}x \, \mathrm{d}y \right) ^{1/p} \end{aligned}$$

where $\Vert \, \, \, \Vert _{h(x)}$ denotes the norm in $E_x$ with respect to h, and the space $L^p(pr_1(E))$ as the completion of $\varGamma (pr_1(E))$ in this norm.

Finally, we define the space

$$\begin{aligned} W^{NL}_{w,1,p} := \{ u \in L^p(E) , \nabla ^{NL}_w u \in L^p(pr_1(E)) \}. \end{aligned}$$

Definition 5

(Adjoint of nonlocal covariant derivative) The adjoint of a nonlocal covariant derivative $\nabla ^{NL}_w$ induced by a covariant derivative $\nabla$ compatible with a positive definite metric h is the operator

${\nabla _w^{NL}}^{ *} :\varGamma (pr_1(E)) \longrightarrow \varGamma (E)$ satisfying

$$\begin{aligned} \langle \nabla ^{NL}_w u,\eta \rangle = (u, {\nabla _w^{NL}}^{ *} \eta ) \end{aligned}$$

$\forall \, u \in \varGamma (E)$, $\forall \eta \in \varGamma (pr_1(E))$, where $(\, \, , \, \,)$ is the $L^2$ scalar product on $\varGamma (E)$ induced by h.

Proposition 1

The operator ${\nabla _w^{NL}}^{ *}$ is defined by

$$\begin{aligned} {\nabla ^{NL}_{w}}^{ *} \eta :x \!\longrightarrow \! \int _{M} w(x,y) \left( \tau _{0,T,\gamma _{y,x}} \, \eta (y,x) - \eta (x,y) \right) \, \mathrm{d}y. \end{aligned}$$

(8)

Proof

See [1]. $\square$

Definition 6

(Nonlocal vector bundle total variation of integrable sections) Let $u \in L^1(E)$ and $w :M \times M \longrightarrow {\mathbb {R}}^{+*}$ be a smooth symmetric function, we define the nonlocal vector bundle total variation $VBTV^{NL}_w(u)$ of u as the quantity

$$\begin{aligned} VBTV^{NL}_w(u) = \sup _{\xi \in K_1} \left( \int _{M} (u, \xi )_h \,\,\, \mathrm{d}M\,\,\, \right) \end{aligned}$$

(9)

where $K_{w,a}$ is the closure in $L^2(E)$ of the set $\mathcal {K}_{w,a}$ defined by

$$\begin{aligned} {\mathcal {K}}_{w,a}:= & {} \{ {\nabla ^{NL}_{w}}^{ *} \eta :\, \eta \in \varGamma (pr_1(E)), \Vert \eta (x,y) \Vert _{h(x)}\nonumber \\&\le a \,\,\, \forall x,y \in M \}. \end{aligned}$$

(10)

We denote by $BV^{NL}_w(E)$ the set of sections $u \in L^1(E)$ such that $VBTV^{NL}_w(u) < + \infty$.

Proposition 2

Assuming that $u \in W^{NL}_{w,1,1}(E)$, we have

$$\begin{aligned} VBTV^{NL}_{w} (u) = \Vert \nabla ^{NL}_w u \Vert _{L^1}, \end{aligned}$$

(11)

the second term corresponding to the $L^1$ norm of the distance (32).

Proof

See [1]. $\square$

3 A Variational Model Encoding Properties of the HVS

3.1 Geometrical Interpretation of Existing Retinex Formulations

In this section, we begin with a brief review on Retinex formulations, following the classification of Bertalmío et al. [6] who distinguish two categories: one-dimensional and two-dimensional Retinex formulations. We detail the expressions of two contributions: the original Retinex formulation of Land and McCann [19] that belongs to the first category, and the KBR formulation of Bertalmío et al. [6] that belongs to the second one. Then, we propose an expression that generalizes both formulations.

3.1.1 One-Dimensional Retinex Formulations

In the Retinex formulation of Land and McCann, the perceived color $L(x)=(L^1(x),L^2(x),L^3(x))$ of an image $u=(u^1,u^2,u^3)$ expressed in the RGB color space at the pixel location x can be written

$$\begin{aligned} L^k(x)= \dfrac{1}{N} \sum _{i=1}^N diff(u^k(x),u^k(y_i)), \qquad k=1,2,3, \end{aligned}$$

(12)

where

$$\begin{aligned}&diff(u^k(x),u^k(y_i))=\sum _{t=0}^{T-1} \varTheta _{\epsilon } \log \dfrac{u^k(\gamma _{y_i,x}(t+1))}{u^k(\gamma _{y_i,x}(t))}\\&\text {with} \quad \varTheta _{\epsilon }(.)= \left\{ \begin{array}{cc} . &{} \text {if} \,\, | . | \, \ge \epsilon \\ \\ 0 &{} \text {otherwise} \end{array} \right. \end{aligned}$$

for N randomly distributed pixels $y_i$ satisfying $u^k(y_i) \ge u^k(x)$ and $\gamma _{y_i,x}$ piecewise linear random paths joining $y_i$ and x subdivided into T points $\gamma _{y_i,x}(0),\ldots ,\gamma _{y_i,x}(T)$, with $\gamma _{y_i,x}(0)=y_i$ and $\gamma _{y_i,x}(T)=x$, and such that

$$\begin{aligned} u^k(\gamma _{y_i,x}(0)) = \max _{t \in \{0...T\}} u^k(\gamma _{y_i,x}(t)). \end{aligned}$$

(13)

Condition (13) means that, along the path $\gamma _{y_i,x}$, the image $u^k$ reaches its highest value at the pixel location $y_i$.

The parameter $\epsilon$ serves as a threshold that guarantees that small intensity changes are not taken into account in the computation of the perceived color. Whereas the use of the threshold enables to disregard changes due to noise in the computation, it can also affect its accuracy as small gradients can indicate objects boundaries as well, especially when dealing with natural images.

The use of random paths provides a good trade-off between an accurate and a fast scan of the image content. The choice of linear piecewise paths in order to scan the image content is due to the particular geometry of the piecewise constant pictures that have been used in the seminal experiments conducted by Land and McCann. However, it turns out that linear piecewise paths are not adapted to natural images in the sense that many paths would be required in order to produce a noise-free output, and this affects the filtering time. Since then, the choice of using linear piecewise paths has been widely questioned and several more recent Retinex formulations have incorporated more computationally efficient paths, e.g., Marini and Rizzi [21] use Brownian paths, whereas Cooper and Baqai [9] use double spirals. However, as pointed out by Bertalmío et al. [6], none of these approaches has been able to prevent the formation of noise and artifacts, suggesting that this is an intrinsic limitation of one-dimensional approaches.

3.1.2 Two-Dimensional Retinex Formulations

A two-dimensional Retinex formulation has been introduced by Horn [16], who reformulated the Retinex as a Poisson equation. This seminal approach has led to many two-dimensional Retinex formulations (see, e.g., [6, 20, 34] for more details). Among them, the KBR approach of Bertalmío et al. [6] is inspired by the mathematical formulation of Retinex proposed by Provenzi et al. [25]. In its general form, KBR is given by

$$\begin{aligned} L^k(x) =&\sum _{y:u^k(y) \ge u^k(x)} w(x,y) \, g \left( \dfrac{u^k(x)}{u^k(y)} \right) \\&+ \sum _{y: u^k(y)<u^k(x)} w(x,y) \end{aligned}$$

where $g :]0,1 ]\longrightarrow ]0,1]$ is a strictly increasing function, and where w is a positive, symmetric and normalized kernel. In the experiments they conducted, the authors chose $g(r)=A \log (r) +1$ for some constant A, and w related to the Gaussian kernel. By this choice of the function g, we have

$$\begin{aligned} L^k(x) =&\sum _{y:u^k(y) \ge u^k(x)} w(x,y) \left[ A \log \left( \dfrac{u^k(x)}{u^k(y)} \right) +1 \right] \nonumber \\&+ \sum _{y: u^k(y)<u^k(x)} w(x,y) \nonumber \\ =&\,\,\, 1+ \sum _{y:u^k(y) \ge u^k(x)} w(x,y) \, A \log \left( \dfrac{u^k(x)}{u^k(y)} \right) \nonumber \\ =&\sum _{y:u^k(y) \ge u^k(x)} w(x,y) \left[ A \log \left( \dfrac{u^k(x)}{u^k(y)} \right) \right. \nonumber \\&\left. +\dfrac{1}{ \sum _{y:u^k(y) \ge u^k(x)} w(x,y)} \right] \end{aligned}$$

(14)

Finally, let us point out that we can rewrite (14) as

$$\begin{aligned} L^k(x)=&\sum _{y:u^k(y) \ge u^k(x)} w(x,y) \, \, \sum _{t=0}^{T-1} \Big [\, A \, [\, \log (u^k(\gamma _{y,x}(t+1)) \nonumber \\&- \log (u^k(\gamma _{y,x}(t)) \, ] + \dfrac{1}{ T \sum _{y:u^k(y) \ge u^k(x)} w(x,y)} \Big ]\end{aligned}$$

(15)

where the curve $\gamma _{y,x}$ joining y and x is subdivided into T points $\gamma _{y,x}(0),\ldots ,\gamma _{y,x}(T)$, with $\gamma _{y,x}(0)=y$ and $\gamma _{y,x}(T)=x$.

As mentioned in the introduction, the KBR formulation has several advantages with respect to the original one: Its computational complexity is much lower and it is less affected by artifacts, noise, or haloes.

3.1.3 Reinterpreting Retinex Formulations

The two Retinex formulations aforementioned involve a discretization of $\nabla \log (u^k)$, where $\nabla$ is the Euclidean gradient, as the difference

$$\begin{aligned} \log (u^k(\gamma _{y,x}(t+1))) - \log (u^k(\gamma _{y,x}(t))) \end{aligned}$$

is the discrete counterpart of

$$\begin{aligned} \left\langle \nabla \log (u^k)(\gamma _{y,x}(t)), \overset{.}{\gamma _{y,x}}(t) \right\rangle , \end{aligned}$$

and we claim that $\nabla \log (u^k)$ can be interpreted as an estimation of the perceptual gradient of an image according to Weber’s law in vision.

Indeed, given a uniform background ${\mathcal {I}}$, Weber’s law states that the following equality holds

$$\begin{aligned} \dfrac{\delta \, {\mathcal {I}}}{{\mathcal {I}}}=c, \end{aligned}$$

(16)

where $\delta {\mathcal {I}}$ is the minimum intensity increment of ${\mathcal {I}}$ to which the human sensitivity distinguish ${\mathcal {I}}$ and ${\mathcal {I}}+\delta {\mathcal {I}}$, and c is a constant.

Hence, Weber’s law shows that the human sensitivity to an intensity increment depends on the intensity of the background. In particular, it shows that human perception is more sensitive to intensity changes in dark backgrounds than in the bright ones. Then, formula (16) can be used to determine equal increments of the perceived brightness ${\mathcal {B}}$ as

$$\begin{aligned} \dfrac{\delta \, {\mathcal {I}}}{{\mathcal {I}}}=c \, \delta {\mathcal {B}} \end{aligned}$$

(17)

from which derives a formula for the perceived brightness by integrating equality (17), i.e.,

$$\begin{aligned} {\mathcal {B}}= \dfrac{1}{c} \log {\mathcal {I}} + b_0 \end{aligned}$$

(18)

where $b_0$ is a constant.

Finally, we obtain an estimation b of the perceived brightness of a color image u from formula (18), given by

$$\begin{aligned} b^k= \dfrac{1}{c^k} \log u^k + b^k_0. \end{aligned}$$

(19)

From formula (19), we deduce that $(1/c^k) \, \nabla \log u^k$ estimates the gradient of the perceived brightness of a color image according to Weber’s law, as we have

$$\begin{aligned} \nabla b^k = \dfrac{1}{c^k} \nabla \log u^k. \end{aligned}$$

(20)

As a consequence, the Retinex formulations aforementioned can be interpreted as follows: The perceived color L(x) at the pixel location x of a color image $u :\varOmega \subset {\mathbb {R}}^2 \longrightarrow {\mathbb {R}}^3$ is the channel-wise averaging ${\mathcal {M}}$ of the perceived difference $diff(u^k(x),u^k(y))$ between the pixel intensity $u^k(x)$ and some other pixel intensities $u^k(y)$ in the image domain $\varOmega$, the perceived difference being given by the integration along a path $\gamma _{y,x}$ joining the two pixels of a quantity $f (\nabla \log u^k)$ approximating the estimation (20) of the gradient of the perceived brightness. In the continuous setting, it gives

$$\begin{aligned} L^k(x)= {\mathcal {M}} \{ diff(u^k(x),u^k(y)) ; y \in \varOmega ' \subset \varOmega \} \end{aligned}$$

(21)

with

$$\begin{aligned} diff(u^k(x),u^k(y))\!=\! \int _{0}^T f ( \langle \nabla \log \, u^k(\gamma _{y,x}(t)), \overset{.}{\gamma _{y,x}}(t) \rangle ) \, \mathrm{d}t. \end{aligned}$$

(22)

For the original Retinex formulation (12), we have

• $\varOmega ' =$ {N randomly distributed pixels y such that $u^k(y) \ge u^k(x)$ and such that for each y there exists a random piecewise linear path $\gamma _{y,x}$ joining y and x and such that the restriction of $u^k$ along $\gamma _{y,x}$ reaches its highest value at y.}

• $f :r \longmapsto \varTheta _{\epsilon } \, r$

• ${\mathcal {M}}= \dfrac{1}{N} \sum _{i=1}^N$.

  For the KBR formulation (15), we have:

• $\varOmega '$= {pixels y such that $u^k(y) \ge u^k(x)$}

• $f :r \longmapsto A \, r+ h$ with

  $$\begin{aligned} h=\dfrac{1}{T \sum _{y \in \varOmega '} w(x,y)}. \end{aligned}$$

• ${\mathcal {M}}= \sum _{y \in \varOmega '} w(x,y).$

3.2 Extension of Retinex Formulations with Covariant Derivatives

3.2.1 Limits of the Retinex Formulations

Formulae (21), (22) show that the gradient of the perceived brightness is at the core of Retinex formulations. However, we claim that the original Retinex formulation (12) and the KBR one (15) suffer from, at least theoretically, several drawbacks in order to reproduce accurately the perceived color image.

First, we showed at the end of the previous section that the estimation of the perceived gradient used in these formulations does not match with the psychophysical estimation $(1/c^k) \, \nabla \log u^k$ in (20). Moreover, this latter assumes that Weber’s law holds, but it is well known that Weber’s law fails at low intensity levels, and we claim that the formulae can then be improved by replacing $(1/c^k) \, \nabla \log (u^k)$ with a more accurate representation of the perceptual gradient, e.g., by taking into account more accurate psychophysical and physiological data.

Secondly, one of the main properties of the HVS that any algorithm devoted to reproduce the perceived color image should take into account is color constancy, i.e., the ability of the HVS to perceive colors almost independently of the lighting of the scene. Whereas existing Retinex formulations succeed in reproducing this ability under particular lighting conditions and properties of the scene, it is not clear that they are able to reproduce it in all cases, especially when natural scenes are involved.

3.2.2 Georgiev’s Reinterpretation of the Perceived Gradient of an Image

Georgiev [14] argues that the ability of the HVS to adapt its perception of scenes to lighting conditions could be related to an invariance property of the perceived gradient with respect to relighting.

Indeed, he suggests that this latter is transformed as the pixel intensities are under a change in lighting conditions, the relighting being mathematically represented by a $3\times 3$ diagonal strictly positive matrix field ${\mathcal {G}}$ acting on the pixel intensities expressed in a trichromatic color space (e.g., LMS or RGB). When the relighting is constant over the image domain, i.e., when ${\mathcal {G}}$ is constant, the standard gradient can serve as a definition of the perceived gradient as the standard differential operator d satisfies $d ({\mathcal {G}} u)= {\mathcal {G}} d u$ (we have replaced $\log u$ by u for simplification). However, it fails to model invariance under nonconstant relighting, as the standard derivative takes into account the variation of the relighting ($d ({\mathcal {G}} u) = d {\mathcal {G}} u + {\mathcal {G}} d u)$. On the other hand, as pointed out by Georgiev, a covariant derivative $\nabla u$ of a section u satisfies an invariance property with respect to a frame change. Indeed, under the frame change given by ${\mathcal {G}}^{-1}$, the following transformations hold

$$\begin{aligned} \left\{ \begin{array}{l} u \longrightarrow {\mathcal {G}} u \\ \omega \longrightarrow {\mathcal {G}} \, d{\mathcal {G}}^{-1} + {\mathcal {G}} \omega {\mathcal {G}}^{-1}. \end{array} \right. \end{aligned}$$

Hence, the expression of the covariant derivative of u in the frame ${\mathcal {G}}^{-1}$ is given by

$$\begin{aligned} \nabla ({\mathcal {G}} \, u)&= d {\mathcal {G}} \, u + {\mathcal {G}} \, du + ({\mathcal {G}} \, d{\mathcal {G}}^{-1} + {\mathcal {G}} \, \omega {\mathcal {G}}^{-1}) {\mathcal {G}} u \\&= {\mathcal {G}} \, d u + {\mathcal {G}} \omega u\\&= {\mathcal {G}} \nabla u, \end{aligned}$$

meaning that the covariant derivative of a section follows the same transformation rule as the section under a frame change. Based on this observation, Georgiev proposes to model the perceived gradient of an image as a covariant derivative, which makes perceived gradients satisfy the same invariance property as the pixel values under a relighting.

3.2.3 The Proposed Extension of the Retinex Formulations

It follows from the analysis performed in Sect. 3.2.2 that a color image $u :\varOmega \subset {\mathbb {R}}^2 \longrightarrow {\mathbb {R}}^3$ can be considered as a section of a G-associated vector bundle of rank 3 over $\varOmega$. When the Lie group G is the set $D^{+*}_3$ of 3x3 diagonal strictly positive matrices, this construction makes a light change be represented by a moving frame change in the vector bundle.

Based on the limitations of the current Retinex formulations mentioned in Sect. 3.2.1, we propose to replace $f(\nabla \log )$, where $\nabla$ is the Euclidean gradient operator, by a covariant derivative $\nabla$ in formula (22), which gives

$$\begin{aligned} diff(u(x),u(y))=\int _{0}^T \nabla _{\overset{\cdot }{\gamma }_{y,x}(t)} u (\gamma _{y,x}(t)) \, \mathrm{d}t \end{aligned}$$

(23)

where $\gamma _{y,x}$ is a curve joining y and x of length T, from which we obtain a new formula for the perceived color image

$$\begin{aligned} L(x)= {\mathcal {M}} \left( \int _{0}^T \nabla _{\overset{\cdot }{\gamma }_{y,x}(t)} u (\gamma _{y,x}(t)) \, \mathrm{d}t \right) \end{aligned}$$

(24)

where ${\mathcal {M}}$ is an averaging operator (not necessarily channel-wise anymore).

The quantity (23) corresponds to

$$\begin{aligned} \tau _{0,T,\gamma _{y,x}}u(y)-u(x) \end{aligned}$$

where $\tau$ denotes the parallel transport map associated with $\nabla$ (see formula (5)).

The introduction of a covariant derivative in formula (23) raises the question of the path dependency of the perceptual difference between two pixel values. To the best of our knowledge, this problem has not been addressed so far.

The dependency of the parallel transport map with respect to the curve joining two points is determined by the curvature of the covariant derivative (see formula (1)). Indeed, if the covariant derivative is flat, then the parallel transport is independent (in a homotopy class) of the curve joining the points, and expression (23) can be simplified as

$$\begin{aligned} diff(u(x),u(y))= {\mathcal {G}}(x) {\mathcal {G}}^{-1}(y) u(y) - u(x) \end{aligned}$$

(25)

where ${\mathcal {G}}$ is the moving frame in which the covariant derivative vanishes.

With respect to the moving frame change from the standard one to ${\mathcal {G}}$, the quantity ${\mathcal {G}}(x) {\mathcal {G}}^{-1}(y) u(y) - u(x)$ rewrites

$$\begin{aligned} {\mathcal {G}}^{-1}(y) u(y) - {\mathcal {G}}^{-1}(x) u(x). \end{aligned}$$

(26)

The fact that ${\mathcal {G}}$ in (26) varies with the pixel location is actually coherent with a lot of color appearance phenomena that make the perceived difference between the colors at two different pixel locations do not correspond to their difference $u(x)-u(y)$, whatever the color space in which the colors are represented is, i.e., whatever the transformation $f(u(x)) - f(u(y))$ (e.g., $f=\log$). Figure 1 shows an example of color appearance phenomena where the perceived difference between u(x) and u(y) is not zero even if $u(x)=u(y)$.

Fig. 1

Color appearance phenomena show that the perceptual difference between two pixel values does not necessarily correspond to their value difference

Fig. 2

The perceptual difference between two pixel intensities may depend on the path joining the two pixels the eye is following. a Path independency of the perceptual difference between two pixel values. b Path dependency of the perceptual difference between two pixel values

Figure 2a shows that this perceptual difference is independent of the path $\gamma ^1_{y,x}$ or $\gamma ^2_{y,x}$ joining y and x the eye follows, which suggests the use of a flat covariant derivative to compute perceptual differences in Fig. 1. The path dependency of the parallel transport map associated with a nonflat covariant derivative suggests that the perceived difference between two pixel values depends on the colors lying on a selected path joining them, which seems to be coherent with human perception as illustrated in Fig. 2b, where the perceptual difference between u(x) and u(y) depends on the path joining x and y. Indeed, according to the path $\gamma ^1_{y,x}$, the perceptual difference is 0 and corresponds actually to the difference of their pixel values. On the other hand, when the eye follows the path $\gamma ^2_{y,x}$, it perceives the same difference as shown in Fig. 1.

Nonetheless, this analysis raises the problem of the choice of the curves $\gamma _{y,x}$ in formula (23) that we will discuss in more detail in the next section.

3.3 From Retinex Formulations to Variational Models for Image Processing

3.3.1 Bertalmío et al.’s Variational Model

The original Retinex (12) and KBR (15) formulations suffer from two main drawbacks when applied as algorithms to correcting camera output images.

First, they are not idempotent, which implies that the algorithms may have to be employed several times in order to provide the desired results, but the number of iterations can hardly be automatized, which made Bertalmío et al. [6] propose to replace the KBR algorithm by a PDE that involves both KBR and the image $u_0$ to be processed

$$\begin{aligned} \dfrac{\partial u^k}{\partial t}(x)= - u^k(x) + L^k(x) - \lambda (u^k(x)-u_0^k(x)). \end{aligned}$$

(27)

The PDE (27) provides an algorithm that performs KBR iteratively but allows to control the output of the algorithm through the parameter $\lambda >0$.

Secondly, they both increase the pixel intensities, meaning that they cannot correct overexposed images. To solve that issue, Bertalmío et al. [6] suggested to modify the KBR formulation (15) by anti-symmetrizing it.

Finally, combining these two modifications of KBR leads to a PDE of the form

$$\begin{aligned} \dfrac{\partial u^k}{\partial t}(x) =&- \lambda (u^k(x)-u_0(x)) - \beta (u^k(x)-1/2) \nonumber \\&+ 2 \int _{\varOmega } w(x,y) S_{\alpha } (u^k(x)-u^k(y)) \, \mathrm{d}y, \end{aligned}$$

(28)

where $S_{\alpha }$ is a differentiable function approximating

$$\begin{aligned} f :z \longmapsto \left\{ \begin{array}{cc} -1 &{} \text {if} \,\, z<0 \\ 0 &{} \text {if} \,\, z=0 \\ 1 &{} \text {if} \,\, z>0 \end{array} \right. \end{aligned}$$

and $\beta >0$ is a parameter.

In [6], the approximation is based on Chebyshev polynomials. In a more recent work [24], a more stable approximation has been proposed based on Bernstein polynomials.

The PDE (28) turns out to be the gradient descent of a differentiable approximation of the variational problem

$$\begin{aligned} \underline{u}^k= \underset{u^k}{\arg \min }&\, \dfrac{\lambda }{2} \int _{\varOmega } \left( u^k(x) - u^k_0(x) \right) ^2 \mathrm{d}x \nonumber \\&\, + \, \dfrac{\beta }{2} \int _{\varOmega } \left( u^k(x) - 1/2 \right) ^2 \mathrm{d}x\, \nonumber \\&- \int _{\varOmega \times \varOmega } w(x,y)\, | u^k(y)-u^k(x) | \, \mathrm{d}x \, \mathrm{d}y. \end{aligned}$$

(29)

The second term of the energy (29) is related to the gray-world hypothesis in visual adaptation (see [23] and references therein for more details). The third term performs (local) contrast enhancement. Both the second and third terms enable to significantly reduce the color cast of the initial image $u_0$, i.e., they tend to make the model (29) perform color constancy.

The aim of the model (29) is to construct an image, given by a solution $\underline{u}$ of the model that provides a better representation of the scene captured by the camera than the camera output image $u_0$. Ideally, the image $\underline{u}$ would be perceived on a screen as the original scene is perceived with the naked eye.

Finally, note that the model is equivalent to

$$\begin{aligned} \underline{u}^k = \underset{u^k}{\arg \min }&\, \dfrac{\lambda +\beta }{2} \int _{\varOmega } \left( u^k(x) \!-\! \dfrac{1}{\lambda +\beta } \, (\lambda u^k_0(x)+ \beta \dfrac{1}{2}) \right) ^2 \mathrm{d}x \nonumber \\&\, - \, \int _{\varOmega \times \varOmega } w(x,y)\, | u^k(y)-u^k(x) | \, \mathrm{d}x \, \mathrm{d}y. \end{aligned}$$

(30)

3.3.2 The Proposed Variational Model

Following the connection established in [6] between the KBR formulation (15) and the variational model (29), we derive the following variational model from the generalization of KBR (24)

$$\begin{aligned} \underline{u}= \underset{u}{\arg \min }&\,\, \dfrac{\lambda +\beta }{2} \int _{\varOmega } \left\| u(x) - \dfrac{1}{\lambda +\beta } \, (\lambda u_0(x)+ \beta v_0) \right\| ^2_{h(x)} \mathrm{d}x \, \nonumber \\&- \, \int _{\varOmega \times \varOmega } w(x,y) \, d^{\tau } (u(x),u(y)) \, \mathrm{d}x \, \mathrm{d}y \end{aligned}$$

(31)

where $v_0$ is the section of coordinates (1 / 2, 1 / 2, 1 / 2) in the standard frame, and the term $d^{\tau }$ is defined by

$$\begin{aligned} d^{\tau } (u(x),u(y))= \, \Vert \tau _{0,T,\gamma _{y,x}} u(y) - u(x) \Vert _{h(x)} \end{aligned}$$

(32)

for some metric h that we interpret as a perceptual distance between the pixels u(x) and u(y). In particular, if the covariant derivative is compatible with the metric h, the perceptual distance (32) is invariant with respect to a moving frame change.

Finally, let us discuss the choice of the kernel w in (31) and the curve $\gamma _{y,x}$ in (32). In [23], the authors consider w as a Gaussian kernel and rely this choice with perception, by taking into account the fact that the strength of “chromatic induction” between two different areas of a scene decreases with their Euclidean distance. This perceptual interpretation of the Euclidean distance between two pixel locations makes us consider in the experiments performed in this paper the straight line as the curve $\gamma _{y,x}$ joining y and x. Note that in most of the experiments we present in Sect. 7, we consider flat covariant derivatives, meaning that the term (32) is independent of the chosen curve $\gamma _{y,x}$. The more complex case of path-dependent perceptual difference, as illustrated in Fig. 2(b), will be discussed in further work.

As in the Euclidean case (29), the solution $\underline{u}$ of the proposed model (31) provides a more accurate representation of the scene captured by the camera than the camera output image $u_0$. Moreover, for a well-chosen covariant derivative, the analysis performed in Sect. 3.2 predicts that it should provide a better representation of the scene than the solution of the Euclidean model as well.

4 A Class of Covariant Derivatives Parametrized by Lie Group Representations

In Sect. 3, we constructed a variational model (31) for color image correction that involves covariant derivatives on a G-associated vector bundle of rank 3, but no explicit covariant derivative was suggested. In Sect. 4, 5, we construct and analyze a class of covariant derivatives, and relate them to some properties of the HVS.

4.1 A Variational Problem for the Construction of Optimal Connection 1-Forms

A covariant derivative on a G-associated vector bundle is completely determined by a ${\mathfrak {g}}$-valued connection 1-form. In what follows, we remind the reader of the construction of optimal connection 1-forms in [2], which is inspired by the Beltrami framework in [28].

The key observation that led to the proposed construction of optimal connection 1-forms was the following. Under the identification between a section S of a G-associated vector bundle and a G-equivariant function $f_S$ on the corresponding principal G-bundle, we have the following correspondence

$$\begin{aligned} d_{_{X^h}} f_S \longleftrightarrow \nabla _{_X} S \end{aligned}$$

where X is a tangent vector field on the base manifold, $X^h$ denotes its horizontal lift with respect to a horizontal distribution on the principal G-bundle (see illustration in Fig. 3), and $\nabla$ is the covariant derivative induced by the horizontal distribution.

Hence, a geometry of color perception, which is given by a covariant derivative on a G-associated vector bundle according to the analysis performed in Sect. 3, is determined by a horizontal distribution of rank 2 on a G-principal bundle, where G is a Lie group acting on a color space.

Fig. 3

Horizontal distribution HP on a G-principal bundle P over M (up) and the induced horizontal lift of a tangent vector field on M at three points $x_1,x_2,x_3$ (down)

Let $I=(I^1,\ldots ,I^n) :\varOmega \subset {\mathbb {R}}^2 \longrightarrow {\mathbb {R}}^n$ be an n-channel image where $\varOmega$ is open that we express as a vector-valued function in the standard basis $(e_1,\ldots ,e_n)$ of ${\mathbb {R}}^n$, i.e., we set $I=I^1 e_1 + \cdots I^n e_n$. Let $(\rho ,G)$ be a group representation on ${\mathbb {R}}^n$ where G is of dimension d and is equipped with a Riemannian metric ${\mathcal {B}}$. Let $(P,\pi _P,\varOmega ,G)$ be the principal G-bundle where $P=\varOmega \times G$ and $\pi _P :\varOmega \times G \longrightarrow \varOmega$ is the standard projection. Let VP be the vertical bundle of P and HP be a horizontal bundle of P.

We assume that P is equipped with a Riemannian metric $Q_1$ constructed as follows: Let $(f_1,f_2)$ be an orthonormal frame of $T\varOmega$ with respect to the Euclidean metric on $\varOmega$. Let $(f_1^h,f_2^h)$ be the horizontal lift of $(f_1,f_2)$ on TP with respect to HP. Let $(X_1,\ldots ,X_d)$ be an orthonormal basis of $T_{e} G = {\mathfrak {g}}$ with respect to ${\mathcal {B}}$. Let $(X_1^v,\ldots ,X_d^v)$ be the frame of VP, defined for $i=1,\ldots ,d$ by

$$\begin{aligned} X_i^v(p)=\left( \dfrac{\mathrm{d}}{\mathrm{d}t} \, p \cdot \exp (t X_i) \right) _{|t=0} \end{aligned}$$

Then, considering the metric $Q_1$ on P given by the Identity matrix field ${\mathbb {I}}_{d+2}$ in the frame $(f_1^h,f_2^h,X_1^v,\ldots ,X_d^v)$ of TP turns $(P,Q_1)$ into a Riemannian manifold.

We consider a second Riemannian metric on P of the form ${\mathbb {I}}_2 \oplus {\mathcal {B}}$ in the frame $(f_1,f_2,X_1^v,\ldots ,X_d^v)$ , from which we derive a Riemannian metric $Q_2$ on $P \times {\mathbb {R}}^n$ of the form ${\mathbb {I}}_2 \oplus {\mathcal {B}} \oplus \kappa \, {\mathbb {I}}_n$, in the frame

$(f_1,f_2,X_1^v,\ldots ,X_d^v,e_1,\ldots ,e_n)$, with $\kappa >0$.

Let J be the $(\rho ,G)$-equivariant function on P given by

$$\begin{aligned} \begin{array}{cccc} &{} P &{}\longrightarrow &{} {\mathbb {R}}^n\\ J :&{} (x,g) &{} \longmapsto &{} \rho (g)^{-1} I(x) \end{array} \end{aligned}$$

(33)

and $\varphi$ its graph, i.e.,

(34)

Then $\varphi$ can be viewed as an embedding of the Riemannian manifold $(P,Q_1)$ into the Riemannian manifold $(P \times {\mathbb {R}}^n, Q_2)$, and we consider its Dirichlet energy $X(\varphi )$ along the section s of P defined by $s(x)=(x,e)$, which is given by

$$\begin{aligned} X(\varphi )= \int _{s} {Q_1}^{\mu \nu } \frac{\partial \varphi ^i}{\partial x_{\mu }} \frac{\partial \varphi ^j}{\partial x_{\nu }} Q_{2{_{ij}}} \,\,\mathrm{d}\varOmega \end{aligned}$$

(35)

where $(x_1,x_2)$ denotes the cartesian coordinates system of $\varOmega$.

Finally, minimizing the energy (35) with respect to the horizontal distribution HP along s, and denoting by $H^{opt}_s P$ the solution, we obtain the restriction along s of an optimal connection 1-form $\omega ^\mathrm{opt}$

$$\begin{aligned} \, H_s^\mathrm{opt}P = :\mathrm{ker} \, \omega ^\mathrm{opt}_{|s}, \end{aligned}$$

and the pulldown $A^\mathrm{opt}$ of $w^\mathrm{opt}$ with respect to s on M given by

$$\begin{aligned} A^\mathrm{opt} :=s^{*} \omega ^\mathrm{opt}. \end{aligned}$$

(36)

Then the connection 1-form $A^\mathrm{opt}$ determines the connection 1-form on the associated vector bundle $P\times _{\rho } {\mathbb {R}}^n$ given by $\rho _{*}(-A^\mathrm{opt})$ in the moving frame of $P \times _{\rho } {\mathbb {R}}^n$ induced by s, i.e., the standard frame $(e_1,\ldots ,e_n)$ of ${\mathbb {R}}^n$.

In what follows, we compute and give the explicit expressions of the connection 1-forms $A^\mathrm{opt}$ (36) and $\rho _{*}(-A^\mathrm{opt})$ for different group representations $(\rho ,G)$, and establish connections with vision.

4.2 The Standard Representation of the Group ${\mathbb {R}}^{+*}$ on ${\mathbb {R}}^n$

4.2.1 Explicit Expression of the Optimal Connection 1-Form

Denoting by a the standard parametrization of the group ${\mathbb {R}}^{+*}$, and equipping it with the metric ${\mathcal {B}}$ given by the constant $\delta$ in the standard frame $\partial /\partial a$, we obtain

$$\begin{aligned} A^\mathrm{opt} = \dfrac{ \sum _{k=1}^n \, I^k \, d I^k}{\delta /\kappa + \sum _{k=1}^n {(I^k)}^2} \otimes \, \partial /\partial a \end{aligned}$$

(37)

Then, the corresponding connection 1-form in the associated bundle $P \times _{\rho } {\mathbb {R}}^n$ is given by

$$\begin{aligned} \rho _{*}(-A^\mathrm{opt})= - \dfrac{ \sum _{k=1}^n \, I^k \, d I^k}{\delta /\kappa + \sum _{k=1}^n {(I^k)}^2} {\mathbb {I}}_n \end{aligned}$$

(38)

in the frame $(e_1,\ldots ,e_n)$.

4.2.2 The Optimal Connection 1-Form for $n=1$ and Its Relation with Weber–Fechner’s Law in Vision

Under the change of variables $J :=I^2,$ the connection 1-form (37) for $n=1$ writes

$$\begin{aligned} A^\mathrm{opt}= \dfrac{1}{2} \dfrac{ d J}{\delta /\kappa + J} \otimes \partial /\partial a, \end{aligned}$$

(39)

and the quantity

$$\begin{aligned} \dfrac{ d J}{\delta /\kappa + J} \end{aligned}$$

in (39) encodes Weber–Fechner’s law in psychophysics.

Indeed, this law, named after the german physicians E. H. Weber and G. Fechner, relates the actual change in a light stimuli and the perceived change. It has been formulated by Fechner, based on a psychophysical experiment conducted earlier by Weber, and is actually a refinement of Weber’s law (16) in the sense that it is more accurate for intensity values ${\mathcal {J}}$ close to 0. More precisely, given an uniform background of intensity ${\mathcal {J}}$, Weber–Fechner’s law claims that the following equality holds

$$\begin{aligned} \dfrac{\delta {\mathcal {J}} }{m + {\mathcal {J}}} = c \end{aligned}$$

(40)

where $\delta {\mathcal {J}}$ is the minimum intensity increment of ${\mathcal {J}}$ to which the human sensitivity distinguish ${\mathcal {J}}$ and ${\mathcal {J}}+\delta {\mathcal {J}}$, c is a constant, $m>0$ is a quantity often interpreted as internal noise in the visual mechanism (its value will be discussed in Sect. 7.2.3).

Weber–Fechner’s law holds for light intensities, while I represents pixel values. However, when displaying I on a screen, the light intensity reaching the eye is known to be approximatively $I^{2.2}$ assuming that $I \in [0,1]$. Hence, for I being a gray-level image, we claim that the quantity

$$\begin{aligned} \dfrac{I \, dI}{\delta /\kappa +{I}^2}, \end{aligned}$$

(41)

for $\delta /\kappa =m$, is an approximation of the perceptual gradient of I (under the identification between the differential and the gradient of a function) that is more accurate than the quantity

$$\begin{aligned} \dfrac{d I}{I} \end{aligned}$$

that appears in Retinex formulations (see Sect. 3.1.3), as this latter is derived from (the less accurate) Weber’s law and acts on pixel values and not on light intensities.

4.3 The Standard Representation of the Group $D^{+*}_3$ on ${\mathbb {R}}^3$

4.3.1 Explicit Expression of the Optimal Connection 1-Form

Denoting by $(a_1,a_2,a_3)$ the standard parametrization of the group $D^{+*}_3$, and equipping it with the metric ${\mathcal {B}}$ given by the diagonal matrix of three strictly positive constant $diag(\delta _1,\delta _2,\delta _3)$ in the standard frame

$(\partial /\partial a_1,\partial /\partial a_2,\partial /\partial a_3)$, we obtain

$$\begin{aligned} A^\mathrm{opt} = \sum _{k=1}^3 \dfrac{ \, I^k \, d I^k}{\delta _k/\kappa + {(I^k)}^2} \otimes \, \partial /\partial a_k. \end{aligned}$$

(42)

Then, the corresponding connection 1-form in the associated bundle $P \times _{\rho } {\mathbb {R}}^3$ is given by

$$\begin{aligned}&\rho _{*}(-A^\mathrm{opt})=\nonumber \\&\quad \left( \begin{array}{ccc} - \dfrac{ \, I^1 \, d I^1}{\delta _1/\kappa + {(I^1)}^2} &{} 0 &{} 0 \\ \\ 0 &{} - \dfrac{ \, I^2 \, d I^2}{\delta _2/\kappa + {(I^2)}^2} &{} 0 \\ \\ 0 &{} 0 &{} - \dfrac{ \, I^3 \, d I^3}{\delta _3/\kappa + {(I^3)}^2} \end{array} \right) \nonumber \\ \end{aligned}$$

(43)

in the frame $(e_1,e_2,e_3)$.

4.3.2 The Optimal Connection 1-Form and Its Relation with Weber–Fechner’s Law in Color Vision

Under the change of variables

$$\begin{aligned} (J^1,J^2,J^3) :=({(I^1)}^2,{(I^2)}^2,{(I^3)}^2), \end{aligned}$$

the connection 1-form (42) writes

$$\begin{aligned} A^\mathrm{opt} = \sum _{k=1}^3 \dfrac{1}{2} \dfrac{ d J^k}{\delta _k/\kappa + J^k} \otimes \, \partial /\partial a_k, \end{aligned}$$

(44)

and the triplet

$$\begin{aligned} \left( \dfrac{ d J^1}{\delta _1/\kappa + J^1}, \dfrac{ d J^2}{\delta _2/\kappa + J^2}, \dfrac{ d J^3}{\delta _3/\kappa + J^3} \right) \end{aligned}$$

(45)

in (44) encodes Weber–Fechner’s law in color.

Indeed, for color images, a natural question concerns the choice of the color space in which the image I has to be expressed. It is known that the three types of cones in the retina (S (short) cones, M (medium) cones, and L (large) cones) have a response that follows Weber–Fechner’s law with respect to the light intensity reaching the eye (assuming that the scene has not a high dynamic range [12]). It suggests that the right color space to consider must be related to the trichromatic theory of colors (also known as Young–Helmholtz theory), which states that the LMS cones are responsible for the color vision in the sense that each of them is sensitive to a different wavelength and that each color can be reproduced as a combination of these three wavelengths. Hence, the color space RGB is a good candidate, as the S resp. M resp. L cones have their peak response in the blue resp. green resp. red wavelength (see, e.g., [7] for more details). However, each type of cone has a different value for the constant c in (40), which implies that the quantity m varies with k and the identification $(\delta _1/\kappa ,\delta _2/\kappa ,\delta _3/\kappa ):=(m^1,m^2,m^3)$ makes the connection 1-form (42) be a good candidate to model the perceptual gradient of a color image. (The values $(m^1,m^2,m^3)$ will be discussed in Sect. 7.2.3.)

4.4 Explicit Expression of the Connection 1-Form for the Standard Representation of SO(2) on ${\mathbb {R}}^2$

Parametrizing SO(2) by an angle $\theta$, and equipping it with the metric ${\mathcal {B}}$ given by a strictly positive constant $\delta$ in the frame $\partial /\partial \theta$, we obtain

$$\begin{aligned} A^\mathrm{opt}= \dfrac{(I^1 d I^2 - I^2 \, d I^1 )}{\delta /\kappa + [{(I^1)}^2 + {(I^2)}^2] }\,\, \otimes \partial /\partial \theta . \end{aligned}$$

(46)

Then, the induced connection 1-form on $P \times _{\rho } {\mathbb {R}}^2$ is given by

$$\begin{aligned}&\rho _{*}(-A^\mathrm{opt}) \nonumber \\&\quad =\left( \begin{array}{cc} 0 &{} \dfrac{ (I^1 d I^2 - I^2 \, d I^1 )}{\delta /\kappa + [{(I^1)}^2 + {(I^2)}^2]} \\ \\ - \dfrac{ (I^1 d I^2 - I^2 \, d I^1 )}{\delta /\kappa + [{(I^1)}^2 + {(I^2)}^2]} &{} 0 \end{array} \right) \nonumber \\ \end{aligned}$$

(47)

in the frame $(e_1,e_2)$.

In polar coordinates, i.e., writing $I^1= r \cos \varphi , I^2=r \sin \varphi$ , the connection 1-form (47) writes

$$\begin{aligned} \rho _{*}(-A^\mathrm{opt}) = \left( \begin{array}{cc} 0 &{} \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \, d\varphi \\ \\ - \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \, d\varphi &{} 0 \end{array} \right) . \end{aligned}$$

(48)

4.5 Representations of SO(2) on ${\mathbb {R}}^3$

4.5.1 Explicit Expression of the Connection 1-Form for a Representation of SO(2) on ${\mathbb {R}}^3$

A group representation $\rho$ of SO(2) on ${\mathbb {R}}^3$ is parametrized by the plane $\mathcal {P}$ in ${\mathbb {R}}^3$ left invariant by the set of rotations $\rho (SO(2))$, and we denote it by $\rho _{_{\mathcal {P}}}$.

Geometric algebra [10] is a powerful tool to represent geometric objects with algebraic expressions, and a plane $\mathcal {P}$ in ${\mathbb {R}}^3$ equipped with the basis $(e_1,e_2,e_3)$ is determined by a bivector $\gamma _{12} \, e_1e_2+ \gamma _{13} \, e_1e_3 + \gamma _{23} \, e_2e_3$, where $\gamma _{12}^2+\gamma _{13}^2+\gamma _{23}^2=1$.

Parametrizing SO(2) by an angle $\theta$, and equipping it with the metric ${\mathcal {B}}$ given by a strictly positive constant $\delta$ in the frame $\partial /\partial \theta$, we obtain

$$\begin{aligned} A^\mathrm{opt} =&\left( \dfrac{ (- \gamma _{12} I^2- \gamma _{13} I^3) \, d I^1}{den} + \dfrac{(\gamma _{12} I^1-\gamma _{23} I^3) \, dI^2}{den} \right. \nonumber \\&\left. + \dfrac{(\gamma _{13} I^1 + \gamma _{23} I^2) \, dI^3}{den}\,\, \right) \otimes \, \partial /\partial \theta \end{aligned}$$

(49)

where

(50)

Writing the 1-form $A^\mathrm{opt}$ as $A_1 \otimes \partial /\partial \theta$, the induced connection 1-form on $P \times _{\rho _{_\mathcal {P}}} {\mathbb {R}}^3$ is given by

$$\begin{aligned} \rho _{_{\mathcal {P}^{*}}}(-A^\mathrm{opt})=-A_1 \left( \begin{array}{ccc} 0 &{} -\gamma _{12} &{} -\gamma _{13} \\ \gamma _{12} &{} 0 &{} -\gamma _{23} \\ \gamma _{13} &{} \gamma _{23} &{} 0 \end{array} \right) \end{aligned}$$

(51)

in the frame $(e_1,e_2,e_3)$.

Let ${\mathcal {F}}=(f_1,f_2,f_3)$ be the orthonormal frame of ${\mathbb {R}}^3$ where $f_2,f_3 \in \mathcal {P}$, given by the matrix

$$\begin{aligned} {\mathcal {F}}= \left( \begin{array}{ccc} -\gamma _{13} \, &{} \, \dfrac{\gamma _{12}+\gamma _{23}}{f} \, &{} \, \dfrac{1- \gamma _{13}(\gamma _{13}-\gamma _{12}+\gamma _{23})}{f} \\ \gamma _{12} \, &{} \, \dfrac{\gamma _{13}-\gamma _{23}}{f} \, &{} \, \dfrac{1-\gamma _{12}(\gamma _{12}-\gamma _{23}-\gamma _{13})}{f}\\ \gamma _{23} \, &{} \, \dfrac{\gamma _{12}+\gamma _{13}}{f} \, &{} \, \dfrac{-1+\gamma _{23}(\gamma _{23}+\gamma _{13}-\gamma _{12})}{f} \end{array} \right) \end{aligned}$$

(52)

in the frame $(e_1,e_2,e_3)$, where

$$\begin{aligned} f=\sqrt{2+2 \, (\gamma _{12}\gamma _{13}+\gamma _{12}\gamma _{23}-\gamma _{13}\gamma _{23})}. \end{aligned}$$

Then, the expression of the connection 1-form

$\rho _{_{\mathcal {P^{*}}}}(-A^\mathrm{opt})$ in the frame ${\mathcal {F}}$ is ${\mathcal {F}}^{-1} \rho _{_{\mathcal {P^{*}}}}(-A^\mathrm{opt}) \, {\mathcal {F}}$, which gives

$$\begin{aligned} \left( \begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} 0 &{} A_1 \\ 0 &{} -A_1 &{} 0 \end{array} \right) . \end{aligned}$$

(53)

Moreover, we can show that $A_1$ can be rewritten as

$$\begin{aligned} A_1= \dfrac{ \, {\mathcal {F}}^{-1}(I)_{2} \, d \, {\mathcal {F}}^{-1}(I)_3 - {\mathcal {F}}^{-1}(I)_3 \, d \, {\mathcal {F}}^{-1}(I)_2}{\delta /\kappa +{\Vert ({\mathcal {F}}^{-1}(I)_{2},{\mathcal {F}}^{-1}(I)_{3}) \Vert }^2}. \end{aligned}$$

(54)

We then observe that the connection 1-form (53) is related to the connection 1-form (47) where $(I^1,I^2)$ is replaced by the projection of $(I^1,I^2,I^3)$ on the plane $\mathcal {P}$, i.e., by $({\mathcal {F}}^{-1}(I)_{2},{\mathcal {F}}^{-1}(I)_{3})$.

Finally, writing $(I^1,I^2,I^3)= (a, r \cos \varphi , r \sin \varphi )$ in the frame ${\mathcal {F}}$, the connection 1-form (53) can be reformulated as

$$\begin{aligned} \rho _{*}(-A^\mathrm{opt}) = \left( \begin{array}{ccc} 0 &{} 0 &{} 0 \\ 0 &{} 0 &{} \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \, d\varphi \\ 0 \,\, &{} \,\, - \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \,\, d\varphi \,\, &{} \, 0 \end{array} \right) . \end{aligned}$$

(55)

4.5.2 Relation with Vision: From Trichromacy to Opponency and Hue Change

As mentioned above, Weber’s and Weber–Fechner’s laws are related to the trichromatic theory of color vision. Color opponency is another theory of color vision that is actually complementary to the trichromatic theory in the sense that it appears at a later stage in the visual pathway. More precisely, whereas the trichromatic theory appears at the retina level, experiments show that the three types of cones and their signals are then transformed in the brain into color opponent mechanisms that have been detected in the lateral geniculate nucleus and the visual cortex. The three opponencies that are physiologically implemented are of the form: white–black, red–green, blue–yellow, and the transformation from the LMS space to the opponency space $O_1O_2O_3$ is given by the $3\times 3$ matrix

$$\begin{aligned} \left( \begin{array}{c} O_1 \\ O_2 \\ O_3 \end{array} \right) = \left( \begin{array}{ccc} 1/\sqrt{3} &{} 1/\sqrt{3} &{} 1/\sqrt{3} \\ 1/\sqrt{2} &{} -1/\sqrt{2} &{} 0 \\ 1/\sqrt{6} &{} 1/\sqrt{6} &{} -2/\sqrt{6} \\ \end{array} \right) \left( \begin{array}{c} L \\ M \\ S \end{array} \right) \end{aligned}$$

(56)

The transformation matrix (56) corresponds to expressing the signal in the basis (52) parametrized by $\gamma _{12}=-\gamma _{13}=\gamma _{23}=1/\sqrt{3}$. Note that the matrix (56) is special orthogonal and makes the white–black, red–green, and yellow–blue axis be orthogonal to each others.

Given a color $O_1O_2O_3$ obtained by the transformation (56) where the LMS space has been replaced by the RGB space, its component $O_1$ corresponds to the intensity of the color, up to a multiplication by the constant $\sqrt{3}$. The opponency color space also exhibits a key notion in color vision, the hue H (see, e.g., [4]), which is defined as $H=arctan(O_3/O_2)$. Finally, the chroma C of the color is $C=\sqrt{O_2^2+O_3^2}$.

We deduce that the group representation of SO(2) on the RGB color space, seen as a subspace of ${\mathbb {R}}^3$, parametrized by $\gamma _{12}=-\gamma _{13}=\gamma _{23}=1/\sqrt{3}$ encodes color transformations that preserve the intensity and chroma, but modify the hue.

4.6 Explicit Expression of the Connection 1-Form for the Standard Representation of SO(3) on ${\mathbb {R}}^3$

Parametrizing SO(3) by the Euler angles $(\theta _1,\theta _2,\theta _3)$, and equipping it with the metric ${\mathcal {B}}$ given by

$$\begin{aligned} \left( \begin{array}{ccc} \delta \, &{} \, 0 \, &{} \, \, \delta \sin \theta _2 \\ 0 \, &{} \, \delta \, &{} \, \, 0 \\ \delta \sin \theta _2 \, &{} \, 0 \, &{} \, \, \delta \end{array} \right) \end{aligned}$$

in the frame $(\partial /\partial \theta _1, \partial /\partial \theta _2,\partial /\partial \theta _3)$ where $\delta >0$, we obtain

$$\begin{aligned} A^\mathrm{opt}&= - \dfrac{ (I^1 d I^2 - I^2 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} \otimes \, \partial /\partial \theta _3 \nonumber \\&\quad + \dfrac{ (I^1 d I^3 - I^3 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} \otimes \, \partial /\partial \theta _2 \nonumber \\&\quad - \dfrac{ (I^2 d I^3 - I^3 dI^2)}{\delta /\kappa + {\Vert I\Vert }^2} \otimes \, \partial /\partial \theta _1. \end{aligned}$$

(57)

Then, the induced connection 1-form on $P \times _{\rho } \, {\mathbb {R}}^3$ is given in the frame $(e_1,e_2,e_3)$ by

$$\begin{aligned}&\rho _{*}(-A^\mathrm{opt})\nonumber \\&\quad =\left( \! \begin{array}{ccc} 0 &{} \dfrac{ (I^1 d I^2 - I^2 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} &{} \dfrac{ (I^1 d I^3 - I^3 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} \\ \\ - \dfrac{ (I^1 d I^2 - I^2 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} &{} 0 &{} \dfrac{ (I^2 d I^3 - I^3 dI^2)}{\delta /\kappa + {\Vert I\Vert }^2} \\ \\ - \dfrac{(I^1 d I^3 - I^3 dI^1)}{\delta /\kappa + {\Vert I\Vert }^2} &{} - \dfrac{(I^2 d I^3 - I^3 dI^2)}{\delta /\kappa + {\Vert I\Vert }^2} &{} 0 \end{array} \right) .\nonumber \\ \end{aligned}$$

(58)

In spherical coordinates, i.e., writing $I^1= r \sin \theta \cos \varphi , I^2= r \, \sin \theta \sin \varphi , \, I^3 = r \, \cos \theta$, the connection 1-form (58) writes

(59)

5 Flatness Property of the Constructed Covariant Derivatives and a New One

In this section, we give the expressions of the curvature 2-forms $F(A^\mathrm{opt})$ of the connection 1-forms $A^\mathrm{opt}$ in (37), (42), (46), (49), (57). Then, we give some conditions on the parameter $\delta /\kappa$ and the image I such that the curvature 2-forms vanish, and from which we determine the moving frames ${\mathcal {G}}$ in which the corresponding connection 1-forms in the associated bundles $P \times _{\rho } {\mathbb {R}}^n$ vanish.

5.1 The Standard Representation of ${\mathbb {R}}^{+*}$ on ${\mathbb {R}}^n$

This is a trivial case as we have the following result.

Proposition 3

The curvature of the connection 1-form $A^\mathrm{opt}$ in formula (37) is 0.

Proof

As ${\mathbb {R}}^{+*}$ is commutative, the term in the expression of the curvature 2-form (see formula (3)) vanishes and the curvature 2-form $F(A^\mathrm{opt})$ corresponds to $dA^\mathrm{opt}$. Then a straightforward computation gives $dA^\mathrm{opt}=0$. $\square$

In the following proposition, we determine a set of moving frames in which the corresponding connection 1-form in the associated bundle $P \times _{\rho } {\mathbb {R}}^n$ vanishes.

Proposition 4

The connection 1-form (38) vanishes in the frames whose matrix representations are

$$\begin{aligned} \mu \sqrt{\delta /\kappa + \sum _{k=1}^n {(I^k)}^2} \,\,\, {\mathbb {I}}_n \end{aligned}$$

(60)

in the frame $(e_1,\ldots ,e_n)$, for $\mu >0$.

Proof

Let ${\mathcal {G}}$ be a moving frame defined in (60). From the change frame formula of a connection 1-form (see expression (2)), a straightforward computation shows that the connection 1-form (38) is 0 in the moving frame ${\mathcal {G}}$. $\square$

5.2 The Standard Representation of $D^{+*}_3$ on ${\mathbb {R}}^3$

This is a trivial case and we have the following result.

Proposition 5

The curvature of the connection 1-form $A^\mathrm{opt}$ given by formula (42) is 0.

Proof

The proof is the same as the one of Prop. 3. $\square$

In the following proposition, we determine a set of moving frames in which the corresponding connection 1-form in the associated bundle $P \times _{\rho } {\mathbb {R}}^3$ vanishes.

Proposition 6

The connection 1-form (43) vanishes in the frames whose matrix representations are

$$\begin{aligned} \left( \begin{array}{ccc} \mu \, {\sqrt{\delta _1/\kappa + {(I^1)}^2}} &{} 0 &{} 0 \\ \\ 0 &{} \mu \, {\sqrt{\delta _2/\kappa + {(I^2)}^2}} &{} 0 \\ \\ 0 &{} 0 &{} \mu \, {\sqrt{\delta _3/\kappa + {(I^3)}^2}} \end{array} \right) \end{aligned}$$

(61)

in the frame $(e_1,e_2,e_3)$, for $\mu >0$.

Proof

The proof is the same as the one of Prop. 4. $\square$

5.3 The Standard Representation of SO(2) on ${\mathbb {R}}^2$

5.3.1 The General Case

Unlike the covariant derivatives induced by the standard representations of ${\mathbb {R}}^{+*}$ on ${\mathbb {R}}^n$ (37) and $D_3^{+*}$ on ${\mathbb {R}}^3$ (42), the covariant derivative induced by the standard representation of SO(2) on ${\mathbb {R}}^2$ (46) is not necessarily flat, as shown in the following proposition.

Proposition 7

The connection 1-form $A^\mathrm{opt}$ in (46) vanishes if and only if $\delta /\kappa \in \{0,\infty \}$ or $d \, I^1 \wedge \, d \, I^2 = 0$.

Proof

As the group SO(2) is commutative, the term vanishes and the curvature $F(A^\mathrm{opt})$ of $A^\mathrm{opt}$ corresponds to $d A^\mathrm{opt}$ (see formula (3)) . Then, computing $dA^\mathrm{opt}$ gives

(62)

$\square$

In the polar form, the curvature writes

$$\begin{aligned} d A^\mathrm{opt} = \dfrac{2 \, \delta /\kappa }{{(\delta /\kappa + \, r^2)}^2} \, r \, d r \wedge d \varphi \otimes \partial /\partial \theta . \end{aligned}$$

(63)

Remark

The quantity $d I^1 \wedge d I^2$ corresponds to the Jacobian of the function $(I^1,I^2)$. Assuming that $\delta /\kappa \notin \{0,\infty \}$, it follows from the theorem of Sard that the curvature $F(A^\mathrm{opt})$ vanishes on $\varOmega$ if and only if there exists F such that $F(I^1(x),I^2(x))=0$ $\forall x \in \varOmega$.

The matrix representation of a smooth orthonormal moving frame ${\mathcal {G}}$ of $P\times _{\rho } {\mathbb {R}}^2$ is of the form

$$\begin{aligned} \left( \begin{array}{cc} \cos \alpha &{} - \sin \alpha \\ \\ \sin \alpha &{} \cos \alpha \end{array} \right) \end{aligned}$$

(64)

in the standard frame $(e_1,e_2)$, for some $\alpha \in C^{\infty }(\varOmega )$.

By the frame change formula (2), the connection 1-form (48) is given by

$$\begin{aligned} \left( \begin{array}{cc} 0 &{} - d \alpha + \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \, d\varphi \\ \\ \, d \alpha - \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \, d\varphi &{} 0 \end{array} \right) . \end{aligned}$$

(65)

in the moving frame ${\mathcal {G}}$ (64).

In what follows, we study some cases where the curvature 2-form given by (62) and (63) vanishes.

5.3.2 The limit cases $\delta /\kappa \in \{0,\infty \}$

Assuming that $\delta /\kappa =0$, it follows from (65) that the moving frame in which the connection 1-form in $P\times _{\rho } {\mathbb {R}}^2$ vanishes is

$$\begin{aligned} \left( \begin{array}{cc} \cos \varphi &{} - \sin \varphi \\ \\ \sin \varphi &{} \cos \varphi \end{array} \right) . \end{aligned}$$

Formula (62) also shows that the connection 1-form (46) is flat for $\delta /\kappa =\infty$. In this case, the connection 1-form in $P \times _{\rho } {\mathbb {R}}^2$ vanishes in the standard frame $(e_1,e_2)$.

5.3.3 The Case r or $\varphi$ is Constant

The case $d \varphi =0$ is trivial as the connection 1-form in $P \times _{\rho } {\mathbb {R}}^2$ vanishes in the standard frame $(e_1,e_2)$ (see expression (48)).

For $dr=0$, we deduce from formula (65) that the moving frame ${\mathcal {G}}$ in which the connection 1-form vanishes satisfies

$$\begin{aligned} \alpha = \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \varphi , \end{aligned}$$

i.e., the matrix representation of ${\mathcal {G}}$ is

$$\begin{aligned} \left( \begin{array}{cc} \cos \left( \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \varphi \right) &{} - \sin \left( \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \varphi \right) \\ \\ \sin \left( \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \varphi \right) &{} \cos \left( \dfrac{ \, r^2}{\delta /\kappa + \,r^2} \varphi \right) \end{array} \right) \end{aligned}$$

(66)

in the standard frame $(e_1,e_2)$.

5.4 Representations of SO(2) on ${\mathbb {R}}^3$

5.4.1 The General Case

The flatness property of the covariant derivative associated with the connection 1-form (49), given by a representation of SO(2) on ${\mathbb {R}}^3$, is induced by the flatness property of the covariant derivative associated with connection 1-form (46), given by the standard representation of SO(2) on ${\mathbb {R}}^2$. Indeed, we have the following result.

Proposition 8

The curvature 2-form of the connection 1-form (49) vanishes if and only if $\delta /\kappa \in \{0,\infty \}$ or $d \, {\mathcal {F}}^{-1}(I)_{2} \wedge d \, {\mathcal {F}}^{-1}(I)_3 =0$.

Proof

As mentioned in Sect. 2.1, a curvature 2-form is transformed in a tensorial way with respect to a frame change. Hence, the curvature 2-form $F(A^\mathrm{opt})$ (49), where $A^\mathrm{opt}=s^{*} \omega ^\mathrm{opt}$, vanishes if there exists $g \in C^{\infty }(\varOmega ;SO(3))$ such that $F((g \cdot s)^{*} \omega ^\mathrm{opt})=0$ vanishes. Then, Prop. 8 is a consequence of formulae (53), (54) and Prop. 7. $\square$

In what follows, we study some cases where the curvature 2-form vanishes. To that purpose, we write I of the form $(a,r \cos \varphi , r \sin \varphi )$ in the frame ${\mathcal {F}}$ (52).

5.4.2 The Limit Cases $\delta /\kappa \in \{0,\infty \}$

We deduce from Sect. 5.3.2 that the connection 1-form given by (51) vanishes in the frame ${\mathcal {F}}$ for $\delta /\kappa =\infty$, and it vanishes in the moving frame whose matrix representation is

$$\begin{aligned} \left( \begin{array}{ccc} 1 \quad &{} 0 &{} 0 \\ \\ 0 \quad &{} \cos \varphi &{} - \sin \varphi \\ \\ 0 \quad &{} \sin \varphi &{} \cos \varphi \end{array} \right) \end{aligned}$$

in the frame ${\mathcal {F}}$ for $\delta /\kappa =0$.

5.4.3 The Case r or $\varphi$ is Constant

We deduce from Sect. 5.3.3 that the connection 1-form given by (51) vanishes in the frame ${\mathcal {F}}$ for $d\, \varphi =0$, and it vanishes in the moving frame whose matrix representation is given by

$$\begin{aligned} \left( \begin{array}{ccc} 1 \quad &{} 0 &{} 0 \\ \\ 0 \quad &{} \cos \left( \dfrac{r^2}{\delta /\kappa + r^2} \varphi \right) &{} - \sin \left( \dfrac{r^2}{\delta /\kappa + r^2} \varphi \right) \\ \\ 0 \quad &{} \sin \left( \dfrac{r^2}{\delta /\kappa + r^2} \varphi \right) &{} \cos \left( \dfrac{r^2}{\delta /\kappa + r^2} \varphi \right) \end{array} \right) \end{aligned}$$

(67)

in the frame ${\mathcal {F}}$ for $dr=0$.

5.4.4 Relation with Hue Maps in Vision

We consider the representation parametrized by

$$\begin{aligned} \gamma _{12}=-\gamma _{13}=\gamma _{23}=1/\sqrt{3} \end{aligned}$$

in the RGB color space.

Then, for $\delta /\kappa =0$, the connection 1-form (51) vanishes in the moving frame ${\mathcal {G}}$ given by

$$\begin{aligned} {\mathcal {G}}= \left( \begin{array}{ccc} 1 \quad &{} 0 &{} 0 \\ \\ \\ 0 \quad &{} \cos H &{} - \sin H \\ \\ \\ 0 \quad &{} \sin H &{} \cos H \\ \end{array} \right) \end{aligned}$$

(68)

in the frame (52), where H is the hue of the image I. Then, connections with vision and neuroscience can be established. Indeed, under the splitting intensity–chrominance, the moving frame (68) exhibits the moving frame

$$\begin{aligned} \left( \begin{array}{cc} \cos H &{} - \sin H \\ \\ \sin H &{} \cos H \end{array} \right) \end{aligned}$$

(69)

in the chrominance plane, which corresponds to the hue field, also called hue maps, introduced by Ben-Shahar and Zucker [4], motivated by the key role of the hue in color vision from both physiological and psychophysical viewpoints. Indeed, on the one hand, neurophysiological findings “imply the existence of neural structures that explicitly encode the hue” and “neurophysiological evidence of hue maps is now emerging.” On the other hand, the hue provides fundamental information about edges of objects in a scene and can then be useful to describe properties of the HVS that are related to edge detection like filling-in phenomena and color constancy.

5.5 The Standard Representation of SO(3) on ${\mathbb {R}}^3$

5.5.1 Explicit Expression of the Curvature

As in the cases of the representations of SO(2) on ${\mathbb {R}}^2$ and ${\mathbb {R}}^3$, the covariant derivative induced by the standard representation of SO(3) on ${\mathbb {R}}^3$ is not necessarily flat. Indeed, we have the following result.

Proposition 9

The curvature of the connection 1-form (57) vanishes if and only if $\delta /\kappa \in \{0,\infty \}$ or

$$\begin{aligned} d I^1 \wedge dI^2 = d I^1 \wedge d I^3 = dI^2 \wedge dI^3 =0. \end{aligned}$$

Proof

Unlike ${\mathbb {R}}^{+*}$ and SO(2), the group SO(3) is not commutative. Hence, the curvature of the connection 1-form $A^\mathrm{opt}$ (57) is $F(A^\mathrm{opt}) := dA^\mathrm{opt} + [A^{opt},A^\mathrm{opt}]$, which gives

(70)

$\square$

Assuming that $\delta /\kappa \notin \{0,\infty \}$, we deduce from the SO(2) case that the curvature vanishes on $\varOmega$ if and only if there exist three functions $F_{12},F_{13},F_{23}$ such that

$$\begin{aligned} F_{12}(I^1,I^2)=F_{13}(I^1,I^3)=F_{23}(I^2,I^3) \equiv 0. \end{aligned}$$

In spherical coordinates, i.e., writing $I^1= r \cos \varphi \sin \theta$, $I^2= r \sin \varphi \sin \theta$, $I^3= r \cos \theta$, the curvature 2-form writes

(71)

In what follows, we study some cases where the covariant derivative is flat and determine the moving frames in which the connection 1-form vanishes.

The limit cases $\delta /\kappa \in \{0,\infty \}$. The case $\delta /\kappa =\infty$ is trivial as formula (58) shows that the connection 1-form vanishes in the standard frame $(e_1,e_2,e_3)$. For $\delta /\kappa =0$, we have the following result.

Proposition 10

The connection 1-form (59) vanishes in the moving frames of the form (106).

Proof

Computing ${\mathcal {G}} ^{-1} d{\mathcal {G}} + {\mathcal {G}} ^{-1} \rho _{*}(-A^\mathrm{opt}){\mathcal {G}}$ for ${\mathcal {G}}$ of the form (106) and $\rho _{*}(-A^\mathrm{opt})$ in (59) gives 0. $\square$

The cases $d\varphi =d\theta =0$. These are trivial cases as the connection 1-form (59) vanishes in the standard frame $(e_1,e_2,e_3)$.

The case $dr=0$. Unlike the SO(2) case in Sect. 5.3, we observe in (71) that the covariant derivative is not necessarily flat when $dr=0$. However, we show in Prop. 11 that the covariant derivative is flat if the pixel values of the image I belongs to a geodesic of $S^2_r$, the sphere of radius r in ${\mathbb {R}}^3$, and we give an explicit expression of a moving frame in which the connection 1-form vanishes.

Recall that the geodesics of $S^2_r$ are the circles of radius r and center (0, 0, 0). A geodesic c of $S^2_r$ is then of the form

$$\begin{aligned} c(t)= r \cos (t) \, u + r \sin (t) \, n \times u \end{aligned}$$

where $n=(\cos \theta \sin \beta ,\sin \theta \sin \beta ,\cos \beta )^T$ is normal to the circle and $u=( \sin \theta , - \cos \theta , 0)^T$ is normal to n.

It follows that $I=(I^1,I^2,I^3)$ is of the form

$$\begin{aligned} \left\{ \begin{array}{l} I^1= r \cos \, \varphi \, \sin \theta + r \sin \varphi \cos \theta \cos \beta \\ I^2= - r \cos \, \varphi \, \cos \theta + r \sin \varphi \cos \beta \sin \theta \\ I^3= - r \sin \varphi \sin \beta \end{array} \right. \end{aligned}$$

(72)

on $S^2_r$, for $\varphi \in C^{1}(\varOmega )$.

We have the following result.

Proposition 11

The covariant derivative induced by the connection 1-form (59) with I of the form (72) is flat. Moreover, the connection 1-form vanishes in the moving frame whose matrix representation ${\mathcal {G}}$ is given by formula (107) in the frame $(e_1,e_2,e_3)$, where

$$\begin{aligned} \alpha = \dfrac{ \, r^2}{\delta /\kappa + r^2} \varphi \end{aligned}$$

Proof

Under the assumption that the connection 1-form (59) is induced by I of the form (72), straightforward computations show that

$$\begin{aligned} d I^1 \wedge d I^2 = d I^1 \wedge d I^3 = d I^2 \wedge d I^3 = 0 \end{aligned}$$

which implies that $F(A^\mathrm{opt})=0$ according to (70).

Then, the matrix representation $\rho _{*}(-A^\mathrm{opt})$ of the connection 1-form is of the form

$$\begin{aligned} \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \left( \begin{array}{ccc} 0 &{} \cos \beta \, d\varphi &{} - \sin \theta \sin \beta \, d\varphi \\ \\ - \cos \beta \, d\varphi &{} 0 &{} \cos \theta \sin \beta \, d\varphi \\ \\ \sin \theta \sin \beta \, d\varphi &{} - \cos \theta \sin \beta \, d\varphi &{} 0 \end{array} \right) \end{aligned}$$

in the standard frame $(e_1,e_2,e_3)$. Finally, applying the frame change formula ${\mathcal {G}} ^{-1} d{\mathcal {G}} + {\mathcal {G}} ^{-1} \rho _{*}(-A^\mathrm{opt}){\mathcal {G}}$ with ${\mathcal {G}}$ of the form (107) gives 0. $\square$

5.5.2 Relation with the Helmholtz–Kohlrausch Effect in Vision

The Helmholtz–Kohlrausch Effect

A consequence of Weber’s and Weber–Fechner’s laws is formulae to approximating the perceived brightness of an image. However, there exist some brightness perception phenomena that these two laws cannot explain, as the Helmholtz–Kohlrausch (H–K) effect. The H–K effect shows that the brightness of a color depends not only on its luminance component, but on the chrominance information (chroma, hue) as well.

In the space CIE $L^{*}a^{*}b^{*}$, Fairchild and Pirrotta [11] developed a formula for the perceived brightness that takes into account the H–K effect and that matches with psychophysical experiments. Denoting by $L^{*}$ the lightness, $C^{*}$ the chroma, and $H^{*}$ the hue components of a color, they define the perceived brightness $L^{**}$ as

$$\begin{aligned} L^{**} = L^{*} + \zeta C^{*} \end{aligned}$$

(73)

where

$$\begin{aligned} \zeta := (2.5 - 0.0025 L^{*} ) \left( 0.116 \left| \, \sin \left( \dfrac{H^{*} - 90}{2} \right) \right| + 0.085 \right) . \end{aligned}$$

From formula (73), we observe two properties of the H–K effect:

1. For given lightness and hue, the brightness of a color is proportional to the chroma. Figure 4 shows the extreme case, i.e., where chromatic colors (i.e., chroma differs from 0) of same lightness but different chrominance are compared to the achromatic color (i.e., chroma equals 0) of same lightness.

Fig. 4

Helmholtz–Kohlrausch effect: chromatic colors appear brighter than achromatic colors

2. For given lightness and chroma, the brightness of a color varies with its hue. For instance, bluish colors appear brighter than the yellowish ones, as illustrated in Fig. 5.

Expressing the H–K Effect in the Vector Bundle Context

The perceived brightness (73) can be interpreted as the $l_1$ norm of the vector $(L^{*}, \zeta \, C^{*})^T$. In order to interpret it as a geometric quantity in the vector bundle context, we replace the $l_1$ norm of the vector $(L^{*}, \zeta \, C^{*})^T$ by its $l_2$ norm that can be written as the $l_2$ norm of the vector $(L^{*},C^{*})$ with respect to the scalar product

$$\begin{aligned} \left( \begin{array}{cc} 1 &{} 0 \\ 0 &{} \zeta ^2 \end{array} \right) \end{aligned}$$

We claim that this replacement does not affect the accuracy of the description of the perceived brightness as the $l_1$ and $l_2$ norms are equivalent.

Fig. 5

Helmholtz–Kohlrausch effect: brightness perception varies the hue. Graphs taken from [11]

Expressing I with its CIE $L^{*}a^{*}b^{*}$ coordinates, and writing $I=L^{*} e_1 + a^{*} e_2 + b^{*} e_3$, its perceived brightness can then be interpreted as its norm with respect to the vector bundle metric given by

$$\begin{aligned} \left( \begin{array}{ccc} 1 &{} 0 &{} 0 \\ \\ 0 &{} \zeta ^2 &{} 0 \\ \\ 0 &{} 0 &{} \zeta ^2 \end{array} \right) \end{aligned}$$

(74)

in the frame $(e_1,e_2,e_3)$, or equivalently its Euclidean norm in the moving frame $(e_1,e_2/\zeta ,e_3/\zeta )$.

Finally, we showed in Sect. 5.5.1 that the connection 1-form is flat and vanishes in the frames ${\mathcal {G}}$ of the form (106) for $\delta /\kappa =0$, where $(r, \varphi ,\theta )$ corresponds to the spherical coordinates of the image I, and a straightforward computation gives ${\mathcal {G}}^{-1} I =(r,0,0)^T$. Then, assuming that I is expressed with its CIE $L^{*}a^{*}b^{*}$ coordinates $(L^{*},a^{*},b^{*})$ in the frame $(e_1,e_2,e_3)$ equipped with the metric (74), we have $r= \sqrt{{L^{*}}^2 + \zeta ^2 {C^{*}}^2}$, which corresponds to the ($l_2$ counterpart of the) color brightness $L^{**}$ (73).

6 Analysis of the Variational Problem Induced by the Constructed Covariant Derivatives and a New One

In this section, we analyze the variational model (31) induced by the connection 1-forms (43), (51), (58), where we suppose that the image I, from which the connection 1-forms are constructed, is the initial image $u_0$. Moreover, we assume that the parameter $\beta$ in (31) is 0, as we aim at analyzing the effect of the variational model on the original image $u_0$. Finally, we construct a new connection 1-form that derives from (43) and (58) and analyze the subsequent variational model.

6.1 On the Solutions of the Variational Model

We study the existence and uniqueness of the solutions of the variational problem (31) for $\beta =0$, which is not straightforward due to the nonconvexity of the variational problem. To that purpose, we follow the standard approach in convex analysis, by considering a dual formulation of the variational problem.

Indeed, we encode the $L_1$ norm of the distance (32) as a generalization of the nonlocal total variation on a vector bundle, as shown in (11). Then, we obtain the following variational problem

$$\begin{aligned} \underset{u \in L^2 \cap BV_w^{NL}(E)}{\arg \min } \, \dfrac{\lambda }{2} {\Vert u - u_0 \Vert }^2_{L^2} - VBTV^{NL}_w (u). \end{aligned}$$

(75)

We showed in [1] the existence of solution(s) in the discrete case, whose expression is (are)

$$\begin{aligned} \underline{u}= u_0 - \underset{u^{*} \in {\mathcal {K}}_{w,1/\lambda }}{\arg \max } \Big \Vert u_0 -u^{*} \Big \Vert ^2_{L^2} \end{aligned}$$

(76)

where the set ${\mathcal {K}}_{w,1/\lambda }$ is defined in formula (10). However, due to the nonconvexity of the problem (76), the uniqueness of the solutions is not guaranteed.

We pointed out in Sect. 3.3 that the perceptual distances (32) are invariant with respect to a moving frame change, provided that the covariant derivative is compatible with the vector bundle metric. Hence, the variational model (31) induced by a flat covariant derivative compatible with the vector bundle metric can be formulated as

$$\begin{aligned} \underline{u} = {\mathcal {G}} \left( \underset{u}{\arg \min } \right.&\, \dfrac{\lambda }{2} \int _{\varOmega } {\Vert u(x) -{\mathcal {G}}^{-1} \, u_0 (x) \Vert }^2 \, dx \nonumber \\&\left. - \int _{\varOmega \times \varOmega } w(x,y) \Vert u(x) - u(y) \Vert \, \mathrm{d}x \, \mathrm{d}y \right) \end{aligned}$$

(77)

where ${\mathcal {G}}$ is the orthonormal moving frame in which the connection 1-form vanishes.

The variational model in (77) is the Euclidean restriction of the model (31) that we studied in [3]. Then, the expression of the solutions of the model (77) in the discrete case is

$$\begin{aligned} \underline{u}=u_0 - {\mathcal {G}} \,\, \underset{u^{*} \in K^0_{w,1/\lambda }}{\arg \max } \, \Vert {\mathcal {G}}^{-1} u_0 - u^{*} \Vert _{L^2} \end{aligned}$$

(78)

where $K^0_{w,1/\lambda }$ is the Euclidean restriction of the convex set $K_{w,1/\lambda }$ defined in (10).

6.2 The Standard Representation of $D^{+*}_3$ on ${\mathbb {R}}^3$

We showed in the previous section that the connection 1-form (43) can be identified with minus the perceptual gradient of a color image according to Weber–Fechner’s law. Based on this observation, we consider from now on the connection 1-form given by minus the connection 1-form (43). Assuming that it is constructed from $u_0$, it gives

$$\begin{aligned} \rho _{*}(A^\mathrm{opt})=\left( \begin{array}{ccc} \dfrac{ \, u_0^1 \, d u_0^1}{\delta _1/\kappa + {(u_0^1)}^2} &{} 0 &{} 0 \\ \\ 0 &{} \dfrac{ \, u_0^2 \, d u_0^2}{\delta _2/\kappa + {(u_0^2)}^2} &{} 0 \\ \\ 0 &{} 0 &{} \dfrac{ \, u_0^3 \, d u_0^3}{\delta _3/\kappa + {(u_0^3)}^2} \end{array} \right) \end{aligned}$$

(79)

in the frame $(e_1,e_2,e_3)$.

A straightforward computation shows that the corresponding covariant derivative is flat as well and that it vanishes in moving frames of the form

$$\begin{aligned} {\mathcal {G}}= diag \left( \left\{ \dfrac{1}{\mu \sqrt{\delta _k/\kappa + {(u_0^k)}^2}} \right\} , \quad k=1,2,3 \right) \end{aligned}$$

(80)

in the standard frame $(e_1,e_2,e_3)$ for $\mu >0$. Moreover, it is compatible with metrics of the form

$$\begin{aligned} h= diag \left( \left\{ \mu ^2 \left[ \delta _k/\kappa + {(u_0^k)}^2 \right] \right\} , \quad k=1,2,3 \right) \end{aligned}$$

(81)

in the standard frame $(e_1,e_2,e_3)$, which means that the generalized distance (32) on $u_0$ can be written

(82)

Finally, the variational model (31) induced by the connection 1-form (79) can be written in the form (77), and the solutions $\underline{u}$ in the form (78).

6.3 The Set of Representations of SO(2) on ${\mathbb {R}}^3$

We consider the variational model (31) associated with the connection 1-form (51) induced by the representation parametrized by

$$\begin{aligned} \gamma _{12}=-\gamma _{13}=\gamma _{23}=1/\sqrt{3} \end{aligned}$$

(83)

in the RGB color space.

The case $\delta /\kappa \in \{0,\infty \}$. For $\delta /\kappa =\infty$, the connection 1-form (51) vanishes in the standard frame $(e_1,e_2,e_3)$, meaning that the variational model (31) is nothing but the Euclidean model we introduced in [3], and which is the vectorial extension of the model (29).

For $\delta /\kappa =0$, ${\mathcal {G}}$ being the moving frame in which the connection 1-form (51) vanishes, and assuming that the space of the representation is the RGB color space, a straightforward computation gives ${\mathcal {G}}^{-1} u_0= ({O_1}_0, C_0, 0)$ where ${O_1}_0$ encodes the intensity component of $u_0$ and $C_0$ stands for its chroma component (see Sect. 4.5.2 for details). Then, it gives

$$\begin{aligned}&d^{\tau } (u_0(x),u_0(y)) =\\&\quad \sqrt{ {({O_1}_0(x)-{O_1}_0(y))}^2+ {({C}_0(x)-{C}_0(y))}^2} \end{aligned}$$

We deduce from (77) that the solutions of the model (31) are of the form

$$\begin{aligned} \underline{u}=(\underline{O_1}, \underline{C} \, \cos H_0, \underline{C} \, \sin H_0) \end{aligned}$$

in the frame ${\mathcal {F}}$ (52), where

$$\begin{aligned}&(\underline{O_1},\underline{C})= \underset{(O_1,C)}{\arg \min } \, \dfrac{\lambda }{2} \int _{\varOmega } {[ (O_1,C)(x) -({O_1}_0,C_0)(x)]}^2 \, \mathrm{d}x \nonumber \\&\quad - \int _{\varOmega \times \varOmega } w(x,y) \, \Vert (O_1,C)(x) - (O_1,C)(y) \Vert \, \mathrm{d}x \, \mathrm{d}y \end{aligned}$$

(84)

Hence, the variational model (31) preserves the hue $H_0$ of the original image $u_0$ and consists in modifying the local contrasts of its intensity and chroma $C_0$ components by processing the vector-valued function $({O_1}_0,C_0)$.

The general case $\delta /\kappa \ne \{0,\infty \}$ In the general case, theconnection 1-form is given by formula (55) in the frame (52) when assuming polar coordinates in the plane left invariant by the rotations. By the representation of SO(2) we consider in this section (83), the differential 1-form

$$\begin{aligned} \dfrac{ \, r^2}{\delta /\kappa + \, r^2} \, d \varphi \end{aligned}$$

in (55) can be viewed as a regularization of the variations of the hue component $H_0$ of $u_0$, the amount of the regularization depending on the chroma $C_0$ of $u_0$ and the constant $\delta /\kappa$. Indeed, the angle $\varphi$ corresponds to $H_0$ and r to $C_0$.

More precisely, for $\delta /\kappa$ well chosen, the variations of the hue are highly reduced for colors with small chroma values (the quantity $C_0^2/(\delta /\kappa + \, C_0^2)$ is small), and almost unchanged for colors with high chroma value (the quantity $C_0^2/(\delta /\kappa + \, C_0^2)$ is close to 1). Reducing the variations of the hue at low chroma values can be very useful for image processing tasks as the hue component is noisy for low chroma values, and especially for low intensity values.

We showed in Sect. 5.4.3 that the covariant derivative is flat if r is constant. Then, assuming that r is constant along the straight line joining the points y and x, we have

$$\begin{aligned} d^{\tau }(u_0(x),u_0(y)) = \Vert {\mathcal {G}} ^{-1}(x) u_0(x) - {\mathcal {G}} ^{-1}(y) u_0(y) \Vert \end{aligned}$$

where ${\mathcal {G}}$ is of the form (67) for $(r,\varphi )=(C_0,H_0)$. It gives

(85)

Then, we observe that $d^{\tau }(u_0(x),u_0(y)) = 0$ if and only if $u_0(x)=u_0(y)$. Moreover, the weight of the component hue $H_0$ is determined by the chroma $C_0$. In particular, we have

$$\begin{aligned} \lim _{C_0 \rightarrow \infty } d^{\tau }(u_0(x),u_0(y)) = \sqrt{ ({O_1}_0(x)-{O_1}_0(y))^2 } \end{aligned}$$

(86)

We deduce that the model (31) tends to preserve regions of constant intensity and chroma assuming that this latter is high enough.

6.4 The Standard Representation of SO(3) on ${\mathbb {R}}^3$

The limit cases $\delta /\kappa \in \{0,\infty \}$ We showed in Sect. 5.5 that the covariant derivative induced by the connection 1-form (58) is flat if $\delta /\kappa \in \{0,\infty \}$.

For $\delta /\kappa =\infty$, the connection 1-form vanishes in the standard Euclidean frame $(e_1,e_2,e_3)$, from which follows that the generalized distance (32) is nothing but the Euclidean distance $\Vert u(x) - u(y) \Vert$ between u(x) and u(y), and the interpretation of this quantity greatly depends on the color space in which the original image $u_0$ is expressed. In particular, assuming that $u_0$ is expressed in its CIE $L^{*}a^{*}b^{*}$ coordinates, the Euclidean distance is an approximation of the perceptual distance between u(x) and u(y) by definition of the CIE $L^{*}a^{*}b^{*}$ color space. As a consequence, the variational model (31) can be interpreted as enhancing an approximation of the local perceived contrast of the original image $u_0$.

For $\delta /\kappa =0$, and assuming that $u_0$ is expressed with its CIE $L^{*}a^{*}b^{*}$ coordinates in the frame $(e_1,e_2,e_3)$ equipped with the metric (74), we showed in Sect. 5.5.2 that ${\mathcal {G}}^{-1} u_0=(L_0^{**},0,0)$, where ${\mathcal {G}}$ is the moving frame in which the connection 1-form vanishes.

As a consequence, the generalized distance (32) on $u_0$ is nothing but the color brightness difference, i.e.,

$$\begin{aligned} d^{\tau }(u_0(x),u_0(y)) = | L_0^{**}(x) - L_0^{**}(y) |. \end{aligned}$$

We deduce from (77) that the solutions $\underline{u}$ of the variational model (31) are of the form

$$\begin{aligned} \underline{u}=(\underline{L}^{**} \cos \varphi _0 \sin \theta _0, \underline{L}^{**} \sin \varphi _0 \sin \theta _0, \underline{L}^{**} \cos \theta _0) \end{aligned}$$

(87)

where

$$\begin{aligned} \underline{L}^{**}=\underset{L^{**}}{\arg \min }&\,\,\, \dfrac{\lambda }{2} \int _{\varOmega } (L^{**}(x)-L_0^{**}(x))^2 \nonumber \\&- \int _{\varOmega \times \varOmega } w(x,y) |L^{**}(x)-L^{**}(y) | \, \mathrm{d}x \, \mathrm{d}y \end{aligned}$$

(88)

i.e., $\underline{L}^{**}$ is solution of the Euclidean model (29).

Hence, the variational model (31) consists in enhancing the local contrast of the color brightness (according to the definition of Fairchild and Pirrotta [11]) of the original $u_0$. Moreover, we deduce from expression (87) that the variational model preserves the hue $H_0^{*}$ of the original image, as $\underline{u}$ corresponds a scaled version of $u_0$ and a straightforward computation shows that the hue is invariant with respect to scalings, and the frame changes from $(e_1,e_2,e_3)$ to $(e_1,e_2/\zeta ,e_3/\zeta )$ and its inverse preserves the hue.

The general case $\delta /\kappa \notin \{0,\infty \}$ In spherical coordinates, the expression of the connection 1-form is given in (59), which can be viewed as a regularization of the variations of the angular components $\theta _0,\varphi _0$ of the original image $u_0$. As the angular components determine the hue of a color, we deduce that, as in the SO(2) case, the connection 1-form is a regularization of the variations of the hue $H_0^{*}$ of $u_0$. Nonetheless, unlike the SO(2) case, the amount of the regularization is not determined by the chroma of $u_0$, but rather by its brightness $L_0^{**}$ (i.e., the radius $r_0$). More precisely, for $\delta /\kappa$ well chosen, the variations of the hue are highly reduced for colors with small brightness values (the quantity $r_0^2/(\delta /\kappa + \, r_0^2)$ is small), and almost unchanged for colors with high brightness values (the quantity $r_0^2/(\delta /\kappa + \, r_0^2)$ is close to 1). Regularizing the hue variations at low brightness values is very useful for image processing tasks as the hue component is noisy in dark areas.

We showed in Sect. 5.5 that the covariant derivative is flat if the values of $u_0$ belong to a geodesic of a sphere in ${\mathbb {R}}^3$, and the moving frame ${\mathcal {G}}$ in which it vanishes is of the form (107). A straightforward computation gives

$$\begin{aligned}&{\mathcal {G}}^{-1} u_0=\\&\quad \left( r_0 \cos \, \left( \dfrac{\delta /\kappa }{\delta /\kappa + r_0^2} \, \varphi _0 \right) , r_0 \sin \, \left( \dfrac{\delta /\kappa }{\delta /\kappa + r_0^2} \, \varphi _0 \right) ,0\right) ^T. \end{aligned}$$

Then, assuming that the covariant derivative is flat along the straight line joining y and x, we have

and

$$\begin{aligned} \lim _{r_0 \rightarrow \infty } d^{\tau } (u_0(x),u_0(y)) = 0, \end{aligned}$$

from which we deduce that the variational model (31) tends to preserve the regions whose colors belong to geodesics of large radii in ${\mathbb {R}}^3$.

6.5 A New Connection 1-Form Associated with the Standard Representation of SO(3) $\times$ $D_3^{+*}$ on ${\mathbb {R}}^3$

Based on the analysis of the connection 1-forms induced by the standard representation of $D_3^{+*}$ on ${\mathbb {R}}^3$ (79) and the standard representation of SO(3) on ${\mathbb {R}}^3$ (58), and the properties of the subsequent variational models, we construct a new connection 1-form associated with the standard representation of the group SO(3) $\times D_3^{+*}$ on ${\mathbb {R}}^3$ in order for the subsequent variational model to satisfy the properties of both variational models.

More precisely, the proposed connection 1-form derives from the following two observations:

(i) The connection 1-form (79) models Weber–Fechner’s law on color images expressed in the RGB color space (see Sect. 4.3.2).

(ii) For $\delta /\kappa =0$, the variational model induced by connection 1-form (58) preserves the hue of the original image $u_0$ (see Sect. 6.4). Note that this analysis has been performed assuming that the image is expressed in the CIE $L^{*}a^{*}b^{*}$ color space equipped with the metric (74), but the same conclusion holds in the RGB color space equipped with the Euclidean metric where the hue is the one of the HSI color space. Actually, in the RGB color space equipped with the Euclidean metric, the model would satisfy the extra property of preserving the saturation, defined as the ration chroma/intensity, of the original image.

In order to construct a connection 1-form that yields a variational model that encodes both properties, we take into account the fact that both covariant derivatives are flat, meaning that their connection 1-forms vanish in some moving frames. Then, the desired connection 1-form is defined as the connection 1-form vanishing in the frame given by the matrix field of the form

$$\begin{aligned} {\mathcal {G}} = {\mathcal {G}}_{(SO3,u_0)} \times {\mathcal {G}}_{(D_3^{+*}, {\mathcal {G}}^{-1}_{(SO3,u_0)}u_0)} \end{aligned}$$

(89)

in the frame $(e_1,e_2,e_3)$, where ${\mathcal {G}}_{(SO3,u_0)}$ is the moving frame in which the connection 1-form (58) constructed from $u_0$ vanishes and which is given by (106), and ${\mathcal {G}}_{(D_3^{+*}, {\mathcal {G}}^{-1}_{(SO3,u_0)}u_0)}$ is the moving frame in which the connection 1-form (79) constructed from ${\mathcal {G}}^{-1}_{(SO3,u_0)}u_0$ vanishes. It gives

$$\begin{aligned}&{\mathcal {G}}_{(D_3^{+*}, {\mathcal {G}}^{-1}_{(SO3,u_0)}u_0)}=\nonumber \\&\quad \left( \begin{array}{ccc} \mu \sqrt{\delta _1/\kappa + r_0^2} &{} 0 &{} 0 \\ \\ 0 &{} \mu \sqrt{\delta _2/\kappa + r_0^2} &{} 0 \\ \\ 0 &{} 0 &{} \mu \sqrt{\delta _3/\kappa + r_0^2} \end{array} \right) . \end{aligned}$$

(90)

Finally, we have

$$\begin{aligned} {\mathcal {G}}^{-1}(u_0)= \left( \mu \, r_0 \sqrt{\delta _1/\kappa + r_0^2}, 0 , 0\right) ^T \end{aligned}$$

and we deduce that the solutions of the model (31) induced by this connection 1-form are of the form

$$\begin{aligned} \underline{u}=(\underline{r} \cos \varphi _0 \sin \theta _0, \underline{r} \sin \varphi _0 \sin \theta _0, \underline{r} \cos \theta _0) \end{aligned}$$

(91)

where $\underline{r}$ is of the form (78) with

$$\begin{aligned} u_0= r_0 \sqrt{\delta /\kappa + (r_0)^2} \end{aligned}$$

and

$$\begin{aligned} {\mathcal {G}}=1/\left( \mu \sqrt{\delta /\kappa + (r_0)^2}\right) . \end{aligned}$$

As expected, we deduce from expression (91) that the hue and the saturation of $u_0$ are preserved as they are invariant with respect to scalings, while a contrast enhancement that follows Weber–Fechner’s law has been applied to $r_0$.

7 Experiments

7.1 On the Numerical Implementation

7.1.1 Numerical Scheme to Reach the Solutions of the Variational Model

As in Sect. 6, we assume that the parameter $\beta$ in the variational problem (31) is 0. In the discrete setting, it is of the form

$$\begin{aligned} \min _{u \in X} \, G(u) - F(A \, u) \end{aligned}$$

(92)

where $A :X \longrightarrow Y$ is a linear operator between two finite-dimensional vector spaces, and possessing an adjoint operator $A^{*}$.

Then, as demonstrated in [30], the problem (92) has the following dual formulation

$$\begin{aligned} \min _{u \in X^{*}} \, (F \circ A)^{*} (u) - G^{*} (u) \end{aligned}$$

(93)

which is equivalent to

$$\begin{aligned} \min _{\eta \in Y^{*}} \, F^{*} (\eta ) - G^{*} (A^{*} \eta ) \end{aligned}$$

(94)

by definition of F.

Hence, the solutions and $\underline{\eta }$ of the primal (92) and dual (94) problems are linked by the following formulae

(95)

(96)

Our proposal is then to adapt the Arrow–Hurwicz approach with fixed step sizes presented in [8] to equations (95) and (96). It gives the following iterative scheme.

$*$ Initialization: Choose $\tau ,\sigma >0$ s.t. $\tau \sigma \Vert A \Vert ^2 < 4$, and

$(u_0,\eta _0) \in L^2(E) \times \varGamma (pr_1(E))$.

$*$ Iterations ($n\ge 0$): Update $u_t,\eta _t$ as follows:

$$\begin{aligned} \left\{ \begin{array}{cc} \eta _{t+1} = &{} \dfrac{\eta _t+\sigma \nabla ^{NL}_w \, u_t}{\max \, \left( 1,\Vert \eta _t+\sigma \nabla ^{NL}_w \, u_t \Vert _h\right) } \\ \\ u_{t+1} = &{} \dfrac{1}{1+ \lambda \tau } \left[ \lambda \tau \, u_0 + \left( u_t+\tau {\nabla ^{NL}_w}^{*} \, \eta _{t+1} \right) \right] \end{array} \right. \end{aligned}$$

(97)

The stopping criteria of the iterative scheme (97), for $u_0$ being a n-channel image, are

$$\begin{aligned} \dfrac{1}{n \, | \varOmega |} \Vert u_{t+1} - u_t \Vert _{L^2} < 0.001. \end{aligned}$$

Due to the nonconvexity of the model (31), the convergence of the numerical scheme (97) is not theoretically guaranteed. However, we would like to point out that, with the fixed parameters $\tau =\sigma =0.1$, it converged in all the experiments we conducted and that we implemented in C++.

Finally, the computation time greatly depends on the flatness of the covariant derivative $\nabla$. Indeed, for a flat covariant derivative, there is no need of computing the parallel transport map in the operators $\nabla ^{NL}_w$ and ${\nabla ^{NL}_w}^{*}$ in (97) as the variational problem restricts to the Euclidean model with respect to the moving frame in which the connection 1-form vanishes. But still, even if the covariant derivative is flat, the computational time is high due to the nonlocality of the model. The proposed strategy to reduce the computational time consists in replacing the Gaussian kernel in the operators $\nabla ^{NL}_w$ and ${\nabla ^{NL}_w}^{*}$ by a kernel of the form

$$\begin{aligned} w(x,y)= \left\{ \begin{array}{cc} \dfrac{1}{| \varOmega ' |} &{} \text {if} \, y \in \varOmega ' \\ \\ 0 &{} \text {otherwise} \end{array} \right. \end{aligned}$$

(98)

where $\varOmega ' \subset \varOmega$ is a square domain containing x. Then, the computational time of the iterative scheme (97) depends on the size of the domain $\varOmega '$.

7.1.2 Numerical Computation of the Parallel Transport Map

Assuming that the covariant derivative $\nabla$ is not flat, there exists no moving frame in which the connection 1-form $\omega$ vanishes, i.e., in which the corresponding parallel transport map is the Identity map. In such a case, the computation of the operators $\nabla ^{NL}_w$ and ${\nabla ^{NL}_w}^{*}$ in (97) requires the computation of the parallel transport map by solving differential equations of the form

$$\begin{aligned} \nabla _{ \overset{.}{\gamma }(t)} \, u(\gamma (t))=0 \end{aligned}$$

(99)

i.e.,

$$\begin{aligned} d u (\gamma (t)) = - \omega ( \overset{.}{\gamma }(t)) (u(\gamma (t))) \end{aligned}$$

(100)

Our proposal is then to make use of an explicit Euler scheme

$$\begin{aligned} u(\gamma (t_{n+1}))=u(\gamma (t_n))-dt \, \omega ( \overset{.}{\gamma }(t_n)) (u(\gamma (t_n))) \end{aligned}$$

(101)

to solve (100) numerically.

The numerical accuracy of the scheme (101) depends on:

(i) The step dt, which is related to the discretization of the straight lines $\gamma$.

(ii) The way we approximate $\omega$ at the points $\gamma (t_n)$ of the discrete lines, as they do not necessarily correspond to pixel locations, which are the points where $\omega$ is constructed.

Let $(x_1,x_2)$ and $(y_1,y_2)$ be two points in $\varOmega$ corresponding to pixel locations, i.e., $x_1,x_2,y_1,y_2 \in {\mathbb {N}}$. We compute the parallel transport of $u(y_1,y_2)$ along the straight line $\gamma _{y,x}$ joining $(y_1,y_2)$ and $(x_1,x_2)$ in the following manner:

(i) We discretize $\gamma$ with $\max (|y_1-x_1|,|y_2-x_2|)$ points. It gives

$$\begin{aligned} dt = \dfrac{1}{\max (|y_1-x_1|,|y_2-x_2|)} \sqrt{(y_1-x_1)^2+(y_2-x_2)^2} \end{aligned}$$

and

$$\begin{aligned} \gamma (t_n)= \left( y_1-n \, \dfrac{(y_1-x_1)}{\max (|y_1-x_1|,|y_2-x_2|)}, \right.&\\ \left. y_2- n \, \dfrac{(y_2-x_2)}{\max (|y_1-x_1|,|y_2-x_2|)} \right) \end{aligned}$$

(ii) We approximate the coefficients

$$\begin{aligned} \omega _{ij}(\overset{.}{\gamma }(t_n)) (\gamma (t_n)) := \overset{.}{\gamma _1}(t_n) \varUpsilon ^j_{1i}(\gamma (t_n)) + \overset{.}{\gamma _2}(t_n) \varUpsilon ^j_{2i}(\gamma (t_n)) \end{aligned}$$

of the matrix $\omega (\overset{.}{\gamma }(t_n))$ at the point $\gamma (t_n)$, for $i,j=1,\cdots ,3$ and $n=0,\cdots ,\max (|y_1-x_1|,|y_2-x_2|)$ in (101) as

$$\begin{aligned} \dfrac{(y_1-x_1)}{\max (|y_1-x_1|,|y_2-x_2|)} \varUpsilon ^j_{1i} \left( R (\gamma (t_n)) \right) +&\\ \dfrac{(y_2-x_2)}{\max (|y_1-x_1|,|y_2-x_2|)} \varUpsilon ^j_{2i} \left( R (\gamma (t_n)) \right) \end{aligned}$$

where $R(\gamma (t_n))$ denotes the rounding of each coordinate of $\gamma (t_n)$ to its nearest integer. In other words, we approximate the symbols $\varUpsilon ^j_{kl},k=1,2,$ of the connection 1-form $\omega$ at the points $\gamma (t_n)$ by $\varUpsilon ^j_{kl}(\widetilde{\gamma (t_n)})$ where $\widetilde{\gamma (t_n)}$ is the closest point to $\gamma (t_n)$ that corresponds to a pixel location.

7.2 Analysis of the Results

We test the model (31) with the covariant derivatives induced by the connection 1-forms (79), (58), and the connection 1-form that is derived from these two ones that we describe in Sect. 6.5.

7.2.1 On the Choice of the Parameters and Its Consequences on the Behavior of the Model

Taking $\beta =0$ greatly affects the color constancy property of the model.

Then, the behavior of the local contrast enhancement model (31) is determined by the parameter $\lambda$ and by the relative size of the domain $\varOmega '$ in (98) with respect to the size of the image domain $\varOmega$. We showed on experiments conducted in [1] that relative small domain sizes privilege texture and noise enhancement, while relative large domain sizes privilege contrast enhancement. In all the experiments conducted in this paper, we consider $\varOmega '$ as a $40 \times 40$ window, which makes the model enhance mainly the contrast as the input images are medium sized images.

Then, the intensity of the enhancement is inversely proportional to the value of the parameter $\lambda$.

7.2.2 Evaluation of the Results

We compare the different results visually and by analyzing their behaviors with respect to the intensity, chroma and hue components in the HSI color space. More precisely, we compute the mean local contrast (MLC) of the intensity component $\underline{u}_i$

$$\begin{aligned} MLC (\underline{u}_i) = \dfrac{1}{| \varOmega |} \sum _{x \in \varOmega } \sum _{y \in \varOmega } w(x,y) | \underline{u}_i(x) - \underline{u}_i(y) | \end{aligned}$$

(102)

and chroma component $\underline{u}_c$

$$\begin{aligned} MLC (\underline{u}_c) = \dfrac{1}{| \varOmega |} \sum _{x \in \varOmega } \sum _{y \in \varOmega } w(x,y) | \underline{u}_c(x) - \underline{u}_c(y) | \end{aligned}$$

(103)

of the output images $\underline{u}$, and we compute the hue shift (HS) of $\underline{u}$, i.e., the mean hue difference with respect to the original image $u_0$

$$\begin{aligned} HS(\underline{u})= \dfrac{1}{| \varOmega |} \sum _{x \in \varOmega } d^{\circ } (H(\underline{u})(x),H(u_0)(x)) \end{aligned}$$

(104)

where $d^{\circ }$ is the angular distance.

7.2.3 The Standard Representation of $D^{+*}_3$ on ${\mathbb {R}}^3$

The model (31) induced by the connection 1-form (79) is parametrized by four scalars: the parameters

$\delta _1/\kappa ,\delta _2/\kappa ,\delta _3/\kappa$ of the connection 1-form, and the parameter $\mu$ of the moving frame (80) in which the connection 1-form vanishes. Following the analysis performed in Sect. 4.3.2, we replace the constant $\delta _k/\kappa$ by $m^k$ in Weber–Fechner’s law (40) for each channel k. In order to get an explicit expression of the constant $m^k$, we follow the approach of Ferradans et al. [12], where Weber–Fechner’s law has been applied in the context of tone mapping.

Given a color image ${\mathcal {I}}$ representing light intensity, Ferradans et al. determine the constant $m^k$ as follows. First, they compute the background intensity ${\mathcal {I}}^k_b$ of the one-channel image ${\mathcal {I}}^k$ as

$$\begin{aligned} {{\mathcal {I}}}^k_b = median ({\mathcal {I}}^k)^{0.5} \, \times mean ({\mathcal {I}}^k)^{0.5} \end{aligned}$$

Then, inspired by the data presented by Valeton and van Norren [31], they estimate the semisaturation constant ${\mathcal {I}}^k_s$, which represents the light level at which the photoreceptor response is half maximal, by

$$\begin{aligned} \log _{10} {{\mathcal {I}}_s^k} = \log _{10} {{\mathcal {I}}_b^k} - 0.37 \left( 4+\log _{10}{{\mathcal {I}}_b^k}\right) +1.9 \end{aligned}$$

Finally, based on results that are presented by Wyszecki and Stiles [32], they compute $m^k$ as

$$\begin{aligned} m^k= 10^{-1.2} \, {{\mathcal {I}}_s^k} \end{aligned}$$

(105)

We observe in formula (105) that the constant $m^k$ is actually image content dependent. In our context, the light intensity of original image $u^k_0$ is approximated by the image ${(u^k_0)}^2$. Hence, we compute the constant $m^k$ in (105) replacing ${\mathcal {I}}^k$ by ${(u^k_0)}^2$, $k=1,2,3$.

In Figs. 6 and 7, we show some results of the proposed model (31) induced by the connection 1-form (79), where we test different values of the parameters $\delta _k/\kappa$. More precisely, we test $\delta _k/\kappa =m^k$ with ${\mathcal {I}}^k={(u^k_0)}^2$ in formula (105) in order to encode Weber–Fechner’s law and we also test $\delta _k/\kappa =0$ in order to encode Weber’s law. Regarding this latter, we actually choose $\delta _k/\kappa = 10^{-6}$ in order to avoid division by 0. We express the original image $u_0$ in the RGB color space as Weber’s and Weber–Fechner’s laws are related to the trichromatic theory of colors (see Sect. 4.3.2). Finally, the parameter $\lambda$ in (31) has been set to 0.1 and the parameter $\mu$ of the moving frame (80) has been empirically set to 700 in Fig. 6 and 1200 in Fig. 7. We compare these models with the model of Bertalmío et al. (29) tested with $\lambda =0.1$ and $\beta =0$. Note that, unlike the approach of the authors who use a differentiable approximation of the model (29) and then reach a solution through the corresponding gradient descent (see [5]), we compute the solution of the original model (29) through the proposed numerical scheme (97) restricted to the Euclidean channel-wise case.

Fig. 6

Contrast enhancement of top left image. Top right: result given by the Euclidean model (29). Bottom row: Results given by the model (31) induced by the connection 1-form (79) encoding Weber’s law (left) and Weber–Fechner’s law (45) (right)

Fig. 7

Contrast enhancement of top left image. Top right: result given by the Euclidean model (29). Bottom row: Results given by the model (31) induced by the connection 1-form (79) encoding Weber’s law (left) and Weber–Fechner’s law (45) (right)

Fig. 8

Contrast enhancement of top left image. Result given by the Euclidean model (29) in the RGB color space (top right) and in the $L^{*}a^{*}b^{*}$ color space (second row, left column). Result given by the model (31) induced by the connection 1-form (58) with the metric (74) encoding the Helmholtz–Kohlrausch effect (73)—last three rows: close-ups of the images on the top two rows. From left to right: original image, Euclidean model (29) in the RGB and $L^{*}a^{*}b^{*}$ color spaces, the model (31) induced by the connection 1-form (58) with the metric (74) encoding the Helmholtz–Kohlrausch effect (73)

Fig. 9

Contrast enhancement of top left image with the model (31) induced by three different connection 1-forms. Top right: the connection 1-form (79). Bottom left: the connection 1-form (58). Bottom right: the connection 1-form concatenating (79) and (58)

Fig. 10

Contrast enhancement of top left image with the model (31) induced by three different connection 1-forms. Top right: the connection 1-form (79). Bottom left: the connection 1-form (58). Bottom right: the connection 1-form concatenating (79) and (58)

We observe in both Figs. 6 and 7 that the proposed models (31) induced by the connection 1-forms encoding Weber’s and Weber–Fechner’s laws have a behavior that follows the properties of these laws as they enhance more the contrast in the dark regions than in the bright ones, while the Euclidean model treats dark and bright regions in the same manner. Indeed, recall that these laws state that the HVS is more sensitive to light changes in dark areas than in the bright ones. In particular, we observe that the noise in the sky (brighter area of the image) has been much less enhanced than in Euclidean case. We also observe that the model induced by Weber’s law enhances more the contrast in dark areas than the one induced by Weber–Fechner’s law, but at the cost of an increase in hue shift (see Tables 1 and 2), and the increase in noise and color artifacts (see Fig. 7). It is coherent with the fact that Weber’s law is known to fail at intensity values close to 0.

Table 1 Contrast enhancement of the top left image in Fig. 6

7.2.4 The Standard Representation of SO(3) on ${\mathbb {R}}^3$

The connection 1-form (58) is parametrized by the constant $\delta /\kappa$.

The limit case $\delta /\kappa = 0$: Enhancement of the local contrast of the color brightness. In Fig. 8, we show the result of the proposed model (31) induced by the connection 1-form (58) for $\delta /\kappa =0$, and where the original image $u_0$ is expressed in its CIE $L^{*}a^{*}b^{*}$ coordinates, and the frame $(e_1,e_2,e_3)$ is equipped with the vector bundle metric (74) so that the distance (32) encodes the color brightness (73) difference. The parameter $\lambda$ in (31) has been set to 0.2.

We compare the model (31) to the channel-wise Euclidean model (29) where the original image $u_0$ is expressed in its RGB and its CIE $L^{*}a^{*}b^{*}$ coordinates for $\beta =0$. In order to make the visual comparisons fair, we select the parameter $\lambda$ in (29) such that the MLCs of the intensity components of the output images are comparable. Table 3 shows that they are comparable when the model (29) is tested with $\lambda =0.075$ in the RGB coordinates and $\lambda =0.2$ in the $L^{*}a^{*}b^{*}$ coordinates.

Table 2 Contrast enhancement of the top left image in Fig. 7

According to the results in Fig. 8, the model (31) provides the best results as it generates less hue shift, which is confirmed by the results in Table 3, and less color artifacts as we can observe in the sky or in the close-up images in Fig. 8.

Comparison of the results for different values of $\delta /\kappa$. In Table 4, we compute the mean over the whole Kodak database [18] (24 images) of the three measures MLC intensity, MLC chroma and HS, for the output images of the model (31) for $\lambda =0.1$, and tested with the connection 1-form (58) for three different values of the parameter $\delta /\kappa : 0,1,+\infty$. Unlike the previous experiments, the model is tested in the RGB color space equipped with the Euclidean metric. The results reveal that the MLCs of the intensity (102) and chroma (103) components have opposite behaviors with respect to the value of $\delta /\kappa$. Indeed, the MLC of the intensity decreases with the value of $\delta /\kappa$, whereas the MLC of the chroma increases with $\delta /\kappa$. Moreover, we observe that the HS increases with $\delta /\kappa$. For $\delta /\kappa =0$, we showed theoretically that the model preserves the hue of the original image, meaning that the mean HS of $1.02^{\circ }$ can be interpreted as numerical errors. It is also worth noting that the behaviors we have just mentioned can actually be observed in (almost) all the 24 images. Indeed, the MLC of the intensity decreases in 20 of the 24 images (in the 4 other images, it does not decrease because the MLC of the intensity for $\delta /\kappa =+\infty$ is higher than the one for $\delta /\kappa =1$), the MLC of the chroma increases in the 24 images, and the HS increases in 23 of the 24 images. (In the other image, it does not increase because the HS for $\delta /\kappa =1$ is higher than the one for $\delta /\kappa =+\infty$).

Table 3 Contrast enhancement of the top left image in Fig. 8

Table 4 Contrast enhancement of Kodak database images [18]

Finally, we claim that the parameter value that provides the best results depends greatly on the user’s preferences. Whereas there exists a consensus about the fact that the HS has to be avoided, there is no consensus about what should be the ratio of enhancement of intensity to enhancement of chroma.

7.2.5 The Standard Representation of SO(3) $\times D_3^{+*}$ on ${\mathbb {R}}^3$.

In Figs. 9 and 10, we test the model (31) induced by the connection 1-form associated with the group SO(3) $\times D_3^{+*}$ described in Sect. 6.5. The model is parametrized by the scalar numbers $\mu$ and $\delta _1/\kappa$ that determine the moving frame (90) in which the connection 1-form vanishes. The original image $u_0$ is expressed in the RGB color space, and the model is tested with $\mu =0.005,\delta _1/\kappa =1$ in Fig. 9 and $\mu =0.0001,\delta _1/\kappa =0.1$ in Fig. 10. We compare the results to the ones obtained by considering the model (31) induced by the connection 1-form (79) tested with the same value for $\mu$ and $\delta _1/\kappa$ and with $\delta _1/\kappa =\delta _2/\kappa =\delta _3/\kappa$, as well as the model (31) induced by the connection 1-form (58). All the models have been tested with $\lambda =0.1$.

Table 5 Contrast enhancement of the top left image in Fig. 9

We observe that the model induced by the connection 1-form constructed in Sect. 6.5 provides the best results, and it fulfills the desired properties that motivated its construction in Sect. 6.5. Indeed, it enables to correct the defects produced by the model induced by (79) and (58) while keeping their desired properties.

More precisely, the model induced by (79) enhances the local contrast in a greater extent in the dark areas than in the bright ones, which is a desirable property when processing underexposed images. However, we observe that it produces color artifacts in the dark areas as well, and that (colored) noise appears in the sky. These two defects arise since the model does not preserve the hue of the original image (see Tables 5, 6).

Table 6 Contrast enhancement of the top left image in Fig. 10

These defects do not appear when testing the model with (58), which is hue preserving. (The hue shift that appears in the last two rows in Table 6 is due to the fact that the hue component gets very noisy in the very dark areas.) However, noise has been enhanced in the sky and the contrast in the dark areas has not been much enhanced, as the model processes dark and bright areas in the same extent.

8 Conclusion

In this paper, we investigated the capacity of Lie group representations and covariant derivatives to model some properties of the human visual system, with a special focus on brightness perception. We showed that some color appearance phenomena, i.e., Weber’s and Weber–Fechner’s laws as well as the Helmholtz–Kohlrausch effect can be modeled through covariant derivatives that are solutions of a variational problem parametrized by a scalar number and by a Lie group representation. This result suggests that the human visual system perceives color brightness in an optimal manner. Another interesting consequence is that the different color appearance phenomena aforementioned only differ by parameters of a single variational problem.

By incorporating these covariant derivatives into a variational model for image processing related to the Retinex theory of color perception, we showed on some examples that the images can be visually improved, the behavior of the model being determined by the property of the color appearance phenomena modeled by the covariant derivative.

Finally, following the idea that increasing the accuracy of the properties of the human visual system incorporated in an image processing technique will increase the visual quality of the output image, further work will be devoted to first construct a covariant derivative that encodes both Weber–Fechner’s law and the Helmholtz–Kohlrausch effect, and then to incorporate more properties of the human visual system.

Appendix: Expressions of Some Moving Frames

$$\begin{aligned} \left( \begin{array}{ccc} \cos \varphi \sin \theta \, &{} \, -g_{11} \sin \varphi - g_{21} \cos \theta \cos \varphi \, &{} \, -g_{12} \sin \varphi - g_{22} \cos \theta \cos \varphi \\ \\ \sin \varphi \sin \theta &{} g_{11} \cos \varphi - g_{21} \sin \varphi \cos \theta &{} g_{12} \cos \varphi - g_{22} \sin \varphi \cos \theta \\ \\ \cos \theta &{} g_{21}\sin \theta &{} g_{22} \sin \theta \end{array} \right) \end{aligned}$$

(106)

with $g_{11},g_{12},g_{21},g_{22} \in C^1(\varOmega )$ satisfying

$$\begin{aligned}&\left\{ \begin{array}{ccc} g_{11}^2+g_{21}^2 &{} = &{} 1 \\ g_{12}^2+g_{22}^2 &{} = &{} 1\\ g_{11}g_{12}+g_{21}g_{22} &{} = &{} 0 \\ g_{11} \, d g_{12}+g_{21} \, dg_{22} &{} = &{} \cos \theta \, d\varphi \end{array} \right. \nonumber \\&\quad \left( \begin{array}{ccc} \sin \theta \cos \alpha + \cos \theta \cos \beta \sin \alpha &{} - \sin \theta \sin \alpha + \cos \theta \cos \beta \cos \alpha &{} \cos \theta \sin \beta \\ \\ - \cos \theta \cos \alpha + \cos \beta \sin \theta \sin \alpha &{} \cos \theta \sin \alpha + \cos \beta \sin \theta \cos \alpha &{} \sin \theta \sin \beta \\ \\ - \sin \beta \sin \alpha &{} - \sin \beta \cos \alpha &{} \cos \beta \end{array} \right) \nonumber \\ \end{aligned}$$

(107)