Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The three dimensional reconstruction of an object is a topic of great interest in many different fields of application: from the digitization of curved documents [12] to the reconstruction of archaeological finds [18]. Other examples come from astronomy for the characterization of properties of planets or other astronomical entities [20, 31, 47]. Facial recognition of individuals [45] is useful for application to security.

This problem has always attracted a great attention because there is still no global method for its resolution under realistic assumptions despite the fact that its formulation is rather simple. The pioneering work of Horn [22] and his activity with the collaborators at MIT [23, 24] produced first formulation of the Shape from Shading (SfS) problem in mathematical terms, via a partial differential equation (PDE) and variational problem. These inspiring works gave rise to many other contributions (see e.g. the two surveys [15, 61] for an extensive list of references).

Several approaches to the SfS problem for classical Lambertian surfaces have been proposed in order to compute a solution. These models mainly belong to two classes: methods based on partial differential equations (PDEs) and optimization methods based on the variational approach. In the first class we can find rather old works based on the method of characteristics and recent works based on the approximation of viscosity solutions for first order Hamilton-Jacobi equations (for a comprehensive presentation of the theory of viscosity solutions we refer the interested reader to the book [4]).

In this work we use the differential approach based on Hamilton-Jacobi equations trying to solve some non-Lambertian models which have been proposed in the literature to overcome some of the limitations of the Lambertian model. It is well known that the classical approach leads to a nonlinear partial differential equation of the first order (of Hamilton-Jacobi type) and it has been shown that this problem is ill-posed even in the framework of viscosity solutions (see the seminal papers by Lions, Rouy and Tourin [30, 43] and also [6, 39]). In fact, there can be many viscosity solutions (no matter which regularity is required for the solutions) unless additional conditions/informations are added to the problem or an a-priori choice is made to compute the maximal solution of the Hamilton-Jacobi equation (see [8, 9, 15]). This explains the growing importance of a generalization of this classical problem in order to obtain uniqueness of the solution while reducing the assumptions on the physical reflectance properties of the objects.

A continuous effort has been made by the scientific community to take into account more realistic reflectance models [2, 3, 42, 56], different scenarios including perspective deformations [1, 11, 34, 38, 49, 57] and/or multiple images of the same object [59, 60]. The images can be taken from the same point of view but with different light sources as in the photometric stereo method [29, 32, 48, 58] or from different points of view but with the same light source as in stereo vision [10]. Recent works have considered more complicated scenarios, e.g. when the light source is not at the optical center under perspective camera projection [26]. It is possible to consider in addition other supplementary issues, as the estimation of the albedo [5, 45, 46, 62] or of the direction of the light source that are usually considered known quantities for the model but in practice are hardly available for real images. Depending on what we know the model has to be adapted leading to a calibrated or uncalibrated problem (see [19, 41, 59, 60] for more details). In this work we will assume that the albedo and the light direction are given.

Our Contribution We want to take into account more realistic models for generic surfaces with nonuniform reflection properties, which means that the light intensity of the image does not depend only on the angle between the outgoing normal to the surface and the light source as in the Lambertian model. In particular, we will focus our attention on two non-Lambertian models under orthographic projection originally proposed by Oren-Nayar [35, 36] and by Phong [37]. These models have been introduced to deal respectively with rough or shiny surfaces and are not well suited for other surfaces such as objects with multiple materials, human skin or glass. Typical examples of rough materials are clay and plaster works whereas bronze and plastic are shiny materials.

We should mention that other authors have contributed to the SfS problem for non-Lambertian surfaces. We mention in particular the contributions in [2], who derived the PDEs associated to several models solving them via a Lax-Friedrichs Sweeping (LFS) method and in [26] where the Hamilton-Jacobi equations based on the Oren-Nayar reflectance model appear in spherical coordinate under perspective camera projection. As we said, here we work in Cartesian coordinate under orthographic projection to derive the Hamilton-Jacobi equations for the above mentioned models under general light directions. Some preliminary results just for the Oren-Nayar problem have appeared in [51] and the Lambertian SfS problem with oblique light direction has been studied in [17]. Extending these results to another non-Lambertian model (the Phong model), we will show that the three models share the same fixed point form so that we can have a unified approach to their analysis and approximation. Moreover, we propose a semi-Lagrangian approximation scheme for that general first order PDE, we give evidence that this scheme converges to the weak solution (in the viscosity sense) of that equation and we compare the performances of this approximation scheme with other finite difference solvers. The scheme is also used to test the models on a number of real and synthetic images in order to understand if the introduction of non-Lambertian models can be really effective.

Organization of the paper In Sect. 2.2 we present an overview of the most relevant non-Lambertian models and derive their Hamilton-Jacobi formulation. In Sect. 2.3 we present the semi-Lagrangian schemes for these equations and shortly the Fast Marching and the Fast Sweeping schemes based on finite difference solver. In Sect. 2.4 we compare these methods and algorithms on a series of benchmarks on synthetic and real images. Finally, we conclude with some comments and future perspectives.

2 Some Non-Lambertian Models for the Orthographic SfS

Let us consider a surface given as a graph \(z = u(\mathbf{x}),\mathbf{x} \in \mathbb{R}^{2}\). We will denote by Ω the region inside the silhouette and we will assume ( just for technical reasons) that Ω is an open and bounded subset of \(\mathbb{R}^{2}\). We assume that u(x) ≥ 0 and the surface is standing on a flat background (hence u(x) = 0 on ∂ Ω). Note that non homogeneous Dirichlet boundary condition like u(x) = g(x) can be easily handled in our approach. The function g(x) will represent the height of the surface at the boundary of the silhouette. Clearly, this is an additional information which in general is not available but can be derived, for example, for rotational surfaces or by symmetry arguments.

It is well known that the Shape-from-Shading problem is described by the image irradiance equation introduced by Bruss [7]

$$\displaystyle{ I(\mathbf{x}) = R(\mathbf{N}(\mathbf{x})), }$$
(2.1)

where I(x) is the normalized brightness of the given grey-value image, N(x) is the unit normal to the surface at the point (x, u(x)) and R(N(x)) is the reflection map giving the value of the light reflection on the surface as a function of its orientation (i.e., of the normal) at each point. Note that a more general formulation of the reflectance function R present in the irradiance equation (2.1) consists of adding a dependence on x too, in order to include several features like e.g. non uniform ambient light depending on some diffuse lights in the ambient (that can be generated by other light sources at finite distance). We will not consider this generalization in this paper.

For the analysis of the different models, it would be useful to introduce a representation of the brightness function I(x) in which we can distinguish different terms representing the contribution of ambient, diffused reflected and specular reflected light. We will write then

$$\displaystyle{ I(\mathbf{x}) = k_{A}I_{A}(\mathbf{x}) + k_{D}I_{D}(\mathbf{x}) + k_{S}I_{S}(\mathbf{x}), }$$
(2.2)

where I A (x), I D (x) and I S (x) are respectively the above mentioned components and k A , k D and k S indicate the percentages of these components such that their sum is equal to 1 (we do not consider absorption phenomena). Note that the diffuse or specular albedo is inside the definition of I D (x) or I S (x), respectively. This will allow to switch on and off the different contributions depending on the model. Let us note that the ambient light term represents light present everywhere in a given scene. As we will see in the following sections, the intensity of diffusely reflected light in each direction is proportional to the cosine of the angle θ i between surface normal and light source direction, without taking into account the point of view of the observer, but another diffuse model (the Oren–Nayar model) will consider it in addition. The amount of specular light reflected towards the viewer is proportional to (cosθ s )α, where θ s is the angle between the ideal (mirror) reflection direction of the incoming light and the viewer direction, α being a constant modelling the specularity of the material. In this way we have a more general model and, dropping the ambient and specular component, we retrieve the Lambertian reflection as a special case. In order to underline the differences, let us briefly sketch the classical Lambertian model (L–model) and two non-Lambertian models: the Oren-Nayar model (ON–model) and the Phong model (PH–model). Our goal in this section is to derive the nonlinear PDEs corresponding to each model.

2.1 Lambertian Model

Let us consider a single light source located at infinity in the direction of the unit vector \(\boldsymbol{\omega }\). For a Lambertian surface, which generates a purely diffuse model, the specular component does not exist. So, the general Eq. (2.2) becomes

$$\displaystyle{ I(\mathbf{x}) = k_{A}I_{A}(\mathbf{x}) + k_{D}I_{D}(\mathbf{x}), }$$
(2.3)

whose diffuse component I D (x) is

$$\displaystyle{ I_{D}(\mathbf{x}) =\gamma _{D}\,\mathbf{N}(\mathbf{x})\cdot \boldsymbol{\omega }, }$$
(2.4)

where γ D is the diffuse albedo. Neglecting the ambient component that can be considered as a constant (i.e. setting k A  = 0), recalling that the sum k A + k D + k S must be equal to 1, we obtain that necessarily k D  = 1 and we can omit it in the following. Then, for a Lambertian surface the image irradiance equation (2.1) becomes

$$\displaystyle{ I(\mathbf{x}) =\gamma _{D}\,\mathbf{N}(\mathbf{x})\cdot \boldsymbol{\omega }, }$$
(2.5)

where we assume to know γ D (in the sequel we suppose uniform albedo and we put γ D  = 1, that is all the points of the surface reflect completely the light that hits them). For Lambertian surfaces [23, 24], just considering an orthographic projection of the scene, we can write the model for SfS via a first order nonlinear PDE which describes the relation between the surface u(x) (our unknown) and the brightness function I(x). The data are the grey-value image I(x), the direction of the light source \(\boldsymbol{\omega }\) and the albedo γ D .

Recalling that the normal to a graph is given by

$$\displaystyle{ \mathbf{N}(\mathbf{x}) = (-\nabla u(\mathbf{x}),1)/\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}}, }$$
(2.6)

we can write (2.1) as

$$\displaystyle{ I(\mathbf{x})\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}} +\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) -\omega _{3} = 0\mbox{ in }\varOmega, }$$
(2.7)

where \(\tilde{\boldsymbol{\omega }}:= (\omega _{1},\omega _{2})\). This is a Hamilton-Jacobi type equation which does not admit in general a regular solution. It is known that the mathematical framework to describe its weak solutions is the theory of viscosity solutions as in [30].

It is important to note that if the light is oblique we have shadows in the image since the object projects its shadow on the flat background. Then, we can divide the image into subdomains

$$\displaystyle{ \varOmega _{l} \equiv \{\mathbf{x}: I(\mathbf{x})> 0\},\qquad \varOmega _{s} \equiv \{\mathbf{x}: I(\mathbf{x}) = 0\}, }$$
(2.8)

which represent respectively the “light” and the “black shadow” regions. Naturally, Ω = Ω l ∪Ω s and we assume for simplicity that the projection of the shadows on the background also falls in Ω.

In Ω l the equation is always the same, whereas in the “shadow” region the surface can have any shape since the model is naturally not able to describe the real surface there. One approach is to deal only with the “light” region setting the equation only on Ω l , however this will require to use oblique boundary conditions (e.g., Neumann boundary conditions) on ∂ Ω l to treat the problem in Ω l because the height there is not known on ∂ Ω l . This can in turn create difficulties in the construction of the numerical algorithm since the curved boundary of Ω l can be nonsmooth and can be efficiently approximated only via a triangulation (which collides with the use of a simple structured grid).

Our approach (see [17] for details) includes the region Ω s in the computation by defining there a virtual surface which replaces the unknown surface corresponding to the “black shadow” region. Conventionally, we will substitute it to the surface generated by the “separation plane” (or “shadow plane”), i.e. the plane separating light from shadow. That plane has the same direction of \(\boldsymbol{\omega }\). This means that in Ω s we have to solve the equation

$$\displaystyle{ (\omega _{1},\omega _{2}) \cdot \nabla u(\mathbf{x}) -\omega _{3} = 0,\quad \mathbf{x} \in \varOmega _{s}. }$$
(2.9)

Note that the irradiance equation coincides with (2.9) since I = 0 in Ω s . Then, we can use the same equation everywhere in Ω avoiding in this way the use of boundary conditions on Ω l , i.e. we can write the global problem as

$$\displaystyle{ \left \{\begin{array}{ll} I(\mathbf{x})\sqrt{1 + \mid \nabla u(\mathbf{x} )\mid ^{2}} + (\omega _{1},\omega _{2}) \cdot \nabla u(\mathbf{x}) -\omega _{3} = 0,&\quad \mathbf{x} \in \varOmega, \\ u(\mathbf{x}) = 0 &\quad \mathbf{x} \in \partial \varOmega. \end{array} \right. }$$
(2.10)

Writing the surface as S(x, z) = z − u(x) = 0 for \(\mathbf{x} \in \varOmega,\;z \in \mathbb{R}\), we can obtain a more compact form for (2.10). In fact, ∇S(x, z) = (−∇u(x), 1) and (2.10) becomes

$$\displaystyle{ \left \{\begin{array}{ll} I(\mathbf{x})\mid \nabla S(\mathbf{x},z)\mid -\nabla S(\mathbf{x},z)\cdot \omega = 0,&\quad \mathbf{x} \in \varOmega,\\ u(\mathbf{x} ) = 0 &\quad \mathbf{x} \in \partial \varOmega. \end{array} \right. }$$
(2.11)

Using the equivalence \(\mid \nabla S(\mathbf{x},z)\mid \equiv \max \limits _{a\in \partial B_{3}(0,1)}\{a \cdot \nabla S(\mathbf{x},z)\}\), we get

$$\displaystyle{ \max _{a\in \partial B_{3}(0,1)}\{\;(I(\mathbf{x})a_{1} -\omega _{1},I(\mathbf{x})a_{2} -\omega _{2},I(\mathbf{x})a_{3}) \cdot \nabla S(\mathbf{x},z)\} =\omega _{3}. }$$
(2.12)

For analytical and numerical reasons it is useful to introduce the exponential Kružkov transform μ v(x) = 1 − e −μ u(x). By this change the variable v(x) will assume values only in [0, 1∕μ] whereas u is in principle unbounded. So the change of variable avoids the risk of an overflow in the approximation. Note that here μ is a free positive parameter without a specific physical meaning, but it is important because varying its value it is possible to modify the slope (the slope increases for increasing values of μ). Clearly, once v is obtained we can always get back to the original surface u simply setting u(x) = −ln(1 −μ v(x))∕μ. By the above approach we can write (2.10) in a fixed point form in the new variable v as

$$\displaystyle{ \left \{\begin{array}{ll} \mu v(\mathbf{x}) =\min \limits _{a\in \partial B_{3}}\{\mathbf{b}^{L}(\mathbf{x},a) \cdot \nabla v(\mathbf{x}) + f^{L}(\mathbf{x},a,v(\mathbf{x}))\},&\qquad \text{for} \ \mathbf{x} \in \varOmega, \\ v(\mathbf{x}) = 0, &\qquad \text{for}\ \mathbf{x} \in \partial \varOmega,\end{array} \right. }$$
(2.13)

where \(\mathbf{b}^{L}:\varOmega \times \partial B_{3}(0,1) \rightarrow \mathbb{R}^{2}\) and \(f^{L}:\varOmega \times \partial B_{3}(0,1) \times [0,1/\mu ] \rightarrow \mathbb{R}\) are defined as

$$\displaystyle\begin{array}{rcl} \mathbf{b}^{L}(\mathbf{x},a):= \frac{1} {\omega _{3}} \left (I(\mathbf{x})a_{1} -\omega _{1},I(\mathbf{x})a_{2} -\omega _{2}\right ),& &{}\end{array}$$
(2.14)
$$\displaystyle\begin{array}{rcl} f^{L}(\mathbf{x},a,v(\mathbf{x})):= -\frac{I(\mathbf{x})a_{3}} {\omega _{3}} (1 -\mu v(\mathbf{x}))\} + 1,& &{}\end{array}$$
(2.15)

where B 3 denotes the unit ball in \(\mathbb{R}^{3}\) and ∂ B 3(0, 1) its boundary.

2.2 Oren-Nayar Model

The diffuse reflectance ON–model [35, 36] is an extension of the previous L-model which explicitly allows to handle rough surfaces. The idea of this model is to represent a rough surface as an aggregation of V-shaped cavities, each with Lambertian reflectance properties (see Fig. 2.1a).

Fig. 2.1
figure 1

Description of the ON–model (Figure adapted from [25]). (a ) Facet model for surface patch dA consisting of many V-shaped Lambertian cavities. (b ) Diffuse reflectance for the ON–model

Full size image

The I D brightness equation for the ON–model [36] is given by

$$\displaystyle{ I_{D}(\mathbf{x}) =\gamma _{D}\cos (\theta _{i})(A + B\sin (\alpha )\tan (\beta )\max [0,\cos (\varphi _{r} -\varphi _{i})]) }$$
(2.16)

where

$$\displaystyle\begin{array}{rcl} A = 1 - 0.5\,\sigma ^{2}(\sigma ^{2} + 0.33)^{-1}& &{}\end{array}$$
(2.17)
$$\displaystyle\begin{array}{rcl} B = 0.45\sigma ^{2}(\sigma ^{2} + 0.09)^{-1}.& &{}\end{array}$$
(2.18)

Note that A and B are two nonnegative constants depending on the statistics of the cavities via the roughness parameter σ that we can imagine to take values between 0 and π∕2, representing the slope of the roughness for the surface considered. In this model (see Fig. 2.1b), θ i represents the angle between the unit normal to the surface N(x) and the light source direction \(\boldsymbol{\omega }\), θ r stands for the angle between N(x) and the observer direction V, φ i is the angle between the projection of the light source direction \(\boldsymbol{\omega }\) and the x 1 axis onto the (x 1, x 2)-plane, φ r denotes the angle between the projection of the observer direction V and the x 1 axis onto the (x 1, x 2)-plane and the two variables α and β are given by

$$\displaystyle{ \alpha =\max \left [\theta _{i},\theta _{r}\right ] and \beta =\min \left [\theta _{i},\theta _{r}\right ]. }$$
(2.19)

For smooth surfaces, we have σ = 0 and the ON–model becomes identical to the L–model. In the particular case \(\boldsymbol{\omega }= \mathbf{V} = (0,0,1)\), or, more precisely, when cos(φ r −φ i ) ≤ 0, the equation simplifies and reduces to a L–model scaled by the coefficient A. This happens for example when the unit vectors \(\boldsymbol{\omega }\) and V are perpendicular so that cos(φ r −φ i ) = −1 or, more in general, when the scalar product between \(\tilde{\boldsymbol{\omega }} = (\omega _{1},\omega _{2})\) and \(\tilde{\mathbf{V}} = (V _{1},V _{2})\) is equal to zero. Therefore the ON–model is more general and flexible than the L–model.

Also for this diffuse model we neglect the ambient component. Then, we get k D  = 1 and, as a consequence, in the general Eq. (2.2) the total light intensity I(x) is equal to the only diffuse component I D (x), in this case described by the Eq. (2.16). Hence, for what follows, we will write I(x) instead of I D (x).

To deal with this equation one has to resolve the min and max operators which appear in (2.16) and (2.19). In general, several cases must be considered but here we just take one to illustrate the technique. Namely, we consider the particular case where the position of the light source \(\boldsymbol{\omega }\) and of the observer V coincide in a general oblique direction (see [50, 52] for the other cases and compare with [26] in order to note that we obtain the same cases). This choice implies max[0, cos(φ i −φ r )] = 1, then defining θ: = θ i  = θ r  = α = β and putting for simplicity the albedo γ D  = 1, the Eq. (2.16) simplifies to

$$\displaystyle{ I(\mathbf{x}) =\cos (\theta )\ \left (A\! +\! B\sin (\theta )^{2}\cos (\theta )^{-1}\right ) }$$
(2.20)

and we arrive to a first order nonlinear Hamilton-Jacobi equation

$$\displaystyle{ (I(\mathbf{x}) - B)(\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}}) + A(\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) -\omega _{3}) + B\frac{(-\tilde{\boldsymbol{\omega }}\cdot \nabla u(\mathbf{x}) +\omega _{3})^{2}} {\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}}} = 0, }$$
(2.21)

where \(\tilde{\boldsymbol{\omega }} = (\omega _{1},\omega _{2})\). Following [51], we write the surface as S(x, z) = z − u(x) = 0, for x ∈ Ω, \(z \in \mathbb{R}\), and ∇S(x, z) = (−∇u(x), 1), so (2.21) becomes

$$\displaystyle{ (I(\mathbf{x}) - B)\vert \nabla S(\mathbf{x},z)\vert + A(-\nabla S(\mathbf{x},z)\cdot \boldsymbol{\omega }) + B\left ( \frac{\nabla S(\mathbf{x},z)} {\vert \nabla S(\mathbf{x},z)\vert }\cdot \boldsymbol{\omega }\right )^{2}\vert \nabla S(\mathbf{x},z)\vert = 0. }$$
(2.22)

Defining d(x, z): = ∇S(x, z)∕ | ∇S(x, z) | and \(c(\mathbf{x},z):= I(\mathbf{x}) - B + B(d(\mathbf{x},z)\cdot \boldsymbol{\omega })^{2}\), using the equivalence \(\vert \nabla S(\mathbf{x},z)\vert \equiv \max _{a\in \partial B_{3}}\{a \cdot \nabla S(\mathbf{x},z)\}\) we get

$$\displaystyle{ \max _{a\in \partial B_{3}}\{c(\mathbf{x},z)\,a \cdot \nabla S(\mathbf{x},z) - A\boldsymbol{\omega } \cdot \nabla S(\mathbf{x},z)\} = 0. }$$
(2.23)

Defining the vector field for the ON-model

$$\displaystyle{ \mathbf{b}^{ON}(\mathbf{x},a):= \frac{1} {A\omega _{3}}\left (c(\mathbf{x},z)a_{1} - A\omega _{1},c(\mathbf{x},z)a_{2} - A\omega _{2}\right ), }$$
(2.24)

introducing the exponential Kružkov transform μ v(x) = 1 − e −μ u(x) as already done for the L–model, we can finally write the Dirichlet problem in the new variable v

$$\displaystyle{ \left \{\begin{array}{ll} \mu v(\mathbf{x}) +\max \limits _{a\in \partial B_{3}}\{ -\mathbf{b}^{ON}(\mathbf{x},a) \cdot \nabla v(\mathbf{x}) + \frac{c(\mathbf{x},z)a_{3}} {A\omega _{3}} (1 -\mu v(\mathbf{x}))\} = 1,&\qquad \mbox{ $\mathbf{x} \in \varOmega $}, \\ v(\mathbf{x}) = 0, &\qquad \mbox{ $\mathbf{x} \in \partial \varOmega $.}\end{array} \right. }$$
(2.25)

Note that the simple homogeneous Dirichlet boundary condition is due to the flat background behind the object but a condition like u(x) = g(x) can also be considered if necessary.

In the particular case when cos(φ r −φ i ) = 0, the Eq. (2.16) simply reduces to

$$\displaystyle{ I(\mathbf{x}) = A\cos (\theta ) }$$
(2.26)

and, as a consequence, the Dirichlet problem in the variable v is equal to (2.25) with c(x, z) = I(x).

2.3 Phong Model for Specular Surfaces

The PH–model introduces a specular component to the brightness function I(x). As we said at the beginning of this section, this can be described in general as the sum I(x) = k A I A (x) + k D I D (x) + k S I S (x), where I A (x), I D (x) and I S (x) are the ambient, diffuse and specular light component, respectively. We will set for simplicity k A  = 0 and represent the diffuse component I D (x) as the Lambertian reflectance model.

The most simple specular model is obtained putting the incidence angle equal to the reflection one and \(\boldsymbol{\omega }\), N(x) and R(x) belong to the same plane. The PH–model is an empirical model that was developed by Phong [37] in 1975. This model describes the specular light component I S (x) as a power of the cosine of the angle between the unit vectors V and R(x) (it is the vector representing the reflection of the light \(\boldsymbol{\omega }\) on the surface), then for the Phong model

$$\displaystyle{ I_{S}^{PH}(\mathbf{x}) =\gamma _{ S}(\mathbf{R}(\mathbf{x}) \cdot \mathbf{V})^{\alpha } }$$
(2.27)

where α expresses the specular reflection characteristics of a material.

Hence, the brightness equation for the PH–model is

$$\displaystyle{ I(\mathbf{x}) = k_{D}\gamma _{D}(\mathbf{N}(\mathbf{x})\cdot \boldsymbol{\omega }) + k_{S}\gamma _{S}(\mathbf{R}(\mathbf{x}) \cdot \mathbf{V})^{\alpha }, }$$
(2.28)

where γ D and γ S represent the diffuse and specular albedo, respectively.

We will illustrate in details the PH–model and the numerical scheme to which we arrive in the case of a general oblique light source \(\boldsymbol{\omega }\) and observer V = (0, 0, 1).

Assuming that N(x) is the bisector of the angle between \(\boldsymbol{\omega }\) and R(x), we obtain

$$\displaystyle{ \mathbf{N}(\mathbf{x}) = \frac{\boldsymbol{\omega }+\mathbf{R}(\mathbf{x})} {\vert \vert \boldsymbol{\omega } + \mathbf{R}(\mathbf{x})\vert \vert }\ \text{which implies}\ \mathbf{R}(\mathbf{x}) = \vert \vert \boldsymbol{\omega } + \mathbf{R}(\mathbf{x})\vert \vert \mathbf{N}(\mathbf{x}) -\boldsymbol{\omega }. }$$
(2.29)

From the parallelogram law, taking into account that \(\boldsymbol{\omega }\), R(x) and N(x) are unit vectors, we can write \(\vert \vert \boldsymbol{\omega } + \mathbf{R}(\mathbf{x})\vert \vert = 2(\mathbf{N}(\mathbf{x})\cdot \boldsymbol{\omega })\), then we can derive the unit vector R(x) as follow:

$$\displaystyle\begin{array}{rcl} \mathbf{R}(\mathbf{x})& =& 2(\mathbf{N}(\mathbf{x})\cdot \boldsymbol{\omega })\mathbf{N}(\mathbf{x})-\boldsymbol{\omega } = 2\left ( \frac{-\tilde{\boldsymbol{\omega }}\cdot \nabla u(\mathbf{x}) +\omega _{3}} {\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}}}\right )\mathbf{N}(\mathbf{x}) - (\omega _{1},\omega _{2},\omega _{3}) \\ & =& \left (\frac{-2\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) + 2\omega _{3}} {1 + \vert \nabla u(\mathbf{x})\vert ^{2}} \right )\left (-\nabla u(\mathbf{x}),1\right ) - (\omega _{1},\omega _{2},\omega _{3}). {}\end{array}$$
(2.30)

For V = (0, 0, 1) we have

$$\displaystyle{ \mathbf{R}(\mathbf{x}) \cdot \mathbf{V} = \frac{-2\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) + 2\omega _{3}} {1 + \vert \nabla u(\mathbf{x})\vert ^{2}} -\omega _{3} = \frac{-2\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) +\omega _{3}(1 -\vert \nabla u(\mathbf{x})\vert ^{2})} {1 + \vert \nabla u(\mathbf{x})\vert ^{2}}. }$$
(2.31)

Then, putting α = 1, Eq. (2.28) becomes

$$\displaystyle{ \begin{array}{ll} I(\mathbf{x})(1 + \vert \nabla u(\mathbf{x})\vert ^{2})& - k_{D}\gamma _{D}(-\nabla u(\mathbf{x}) \cdot \boldsymbol{\omega } +\omega _{3})(\sqrt{1 + \vert \nabla u(\mathbf{x} )\vert ^{2}}) \\ & - k_{S}\gamma _{S}\left (-2\tilde{\boldsymbol{\omega }} \cdot \nabla u(\mathbf{x}) +\omega _{3}(1 -\vert \nabla u(\mathbf{x})\vert ^{2})\right ) = 0, \end{array} }$$
(2.32)

to which we add a Dirichlet boundary condition equal to zero assuming that the surface is standing on a flat background. As we have done for the previous models, we write the surface as S(x, z) = z − u(x) = 0, for x ∈ Ω, \(z \in \mathbb{R}\), and ∇S(x, z) = (−∇u(x), 1), so (2.32) will be written as

$$\displaystyle{ \begin{array}{ll} I(\mathbf{x})\vert \nabla S(\mathbf{x},z)\vert ^{2} & - k_{D}\gamma _{D}(\nabla S(\mathbf{x},z)\cdot \boldsymbol{\omega })(\vert \nabla S(\mathbf{x},z)\vert )\\ \\ & - k_{S}\gamma _{S}(2\nabla S(\mathbf{x},z) \cdot \boldsymbol{\omega }-\vert \nabla S(\mathbf{x},z)\vert ^{2}\omega _{3}) = 0. \end{array} }$$
(2.33)

Dividing both the terms by | ∇S(x, z) | , defining d(x, z): = ∇S(x, z)∕ | ∇S(x, z) | as in the ON–model and c(x): = I(x) +ω 3 k S γ S , we get

$$\displaystyle{ c(\mathbf{x})\vert \nabla S(\mathbf{x},z)\vert - k_{D}\gamma _{D}(\nabla S(\mathbf{x},z)\cdot \boldsymbol{\omega }) - 2k_{S}\gamma _{S}(d(\mathbf{x},z)\cdot \boldsymbol{\omega }) = 0. }$$
(2.34)

By the equivalence \(\vert \nabla S(\mathbf{x},z)\vert \equiv \max _{a\in \partial B_{3}}\{a \cdot \nabla S(\mathbf{x},z)\}\) we obtain

$$\displaystyle{ \max _{a\in \partial B_{3}}\{c(\mathbf{x})\,a \cdot \nabla S(\mathbf{x},z) - k_{D}\gamma _{D}(\boldsymbol{\omega }\cdot \nabla S(\mathbf{x},z)) - 2k_{S}\gamma _{S}(d(\mathbf{x},z)\cdot \boldsymbol{\omega })\} = 0. }$$
(2.35)

Defining the vector field

$$\displaystyle{ \mathbf{b}^{PH}(\mathbf{x},a):= \frac{1} {Q^{PH}(\mathbf{x},z)}\left (c(\mathbf{x})a_{1} - k_{D}\gamma _{D}\omega _{1},c(\mathbf{x})a_{2} - k_{D}\gamma _{D}\omega _{2}\right ) }$$
(2.36)

where

$$\displaystyle{ Q^{PH}(\mathbf{x},z):= 2k_{ S}\gamma _{S}(d(\mathbf{x},z)\cdot \boldsymbol{\omega }) + k_{D}\gamma _{D}\omega _{3}, }$$
(2.37)

and using the exponential Kružkov transform μ v(x) = 1 − e −μ u(x) as done for the previous models, we can finally write the nonlinear problem corresponding to the PH–model

$$\displaystyle{ \left \{\begin{array}{ll} \mu v(\mathbf{x}) +\max \limits _{a\in \partial B_{3}}\{ -\mathbf{b}^{PH}(\mathbf{x},a) \cdot \nabla v(\mathbf{x}) + \frac{c(\mathbf{x})a_{3}} {Q^{PH}(\mathbf{x},z)}(1 -\mu v(\mathbf{x}))\} = 1,&\qquad \mbox{ $\mathbf{x} \in \varOmega $}, \\ v(\mathbf{x}) = 0, &\qquad \mbox{ $\mathbf{x} \in \partial \varOmega $.} \end{array} \right. }$$
(2.38)

Again, note that the simple homogeneous Dirichlet boundary condition considered is due to the flat background behind the object but a different boundary condition can also be considered.

3 Numerical Approximation

Let us describe some numerical schemes for the solution of the problems described in the previous section. Here we will focus our attention on semi-Lagrangian (SL) schemes which have shown to be very effective for first order problems since they try to mimic at the discrete level the method of characteristics (see [16] for more details). Other approaches based on finite differences or finite volumes are feasible. As we have seen there are basically two main problems related to the vertical light case and the oblique light case. In the vertical case, we have to solve an eikonal-type equation for each model. In the oblique case, we get the more general first-order Hamilton-Jacobi (HJ) equations (2.7), (2.21), and (2.32) where the nonlinear term is also coupled with linear terms. The general framework for these type of problems is the theory of viscosity solutions which guarantees (under appropriate assumptions) existence and uniqueness results for the vertical light case. A similar approach can also be applied to the case of an oblique light source when the surface is not smooth and black shadows are present in the image [17]. It should be noted that to have uniqueness when the eikonal equation is degenerate (i.e. when the right-hand side vanishes at some points) one has to add additional assumptions or more informations (like the height at maximum brightness points or the fact that we select to approximate the maximal solution, as introduced in [8]). General convergence results for the approximation scheme to the maximal solution of the degenerate eikonal equation can be found in [9, 15].

There are two types of algorithms based on the semi-Lagrangian approach. The first type of algorithm is global and gives an approximation of the fixed point problem on the whole grid at every iteration till the stopping rule is satisfied. Some acceleration methods, like the Fast Sweeping method [27, 28], can be introduced to speed up convergence. The second type of method is local and tries to concentrate the numerical effort only in a neighborhood of a region which is considered to be already exact (the so called Accepted region). The Fast Marching method (extensively described in [13, 44]) is a typical example of this class of methods.

The algorithms corresponding to the models presented in the previous section compute the maximal solution in the domain without additional information of the surface and with a single boundary condition which can be either homogeneous u = 0 or not (but to set u = g on the boundary of the mask one has to know or guess the right solution there). This is due to the monotonicity properties of the discrete operator corresponding to the schemes. The interested reader can find in [16] a detailed presentation of the properties of semi-Lagrangian schemes and in [17] an application to the Shape-from-Shading problem with black shadows.

As already stated in Sect. 2.2, we suppose a surface given as a graph. In the case of vertical light, for such a surface we do not have shadows covering an open domain (i.e. the points where I(x) = 0 are either isolated or curves in the plane). If the light is oblique, we usually have shadows so that we can divide the support of the surface (the domain of u) into two regions, Ω l  ≡ {x: I(x) > 0} and Ω s  ≡ {x: I(x) = 0}, which represent respectively the “light” and the “shadow” regions. Typically they have both nonempty interior and, naturally, Ω = Ω l ∪Ω s . Note that Ω now represents the new mask which also includes black regions. Moreover, we assume that \(\varOmega \subset \overline{Q}\), where Q is the rectangular domain corresponding to the image.

As we already explained, we can use the same equation everywhere in our computational domain Q and we do not need to introduce any boundary condition on ∂ Ω l (see [17] for more details).

Now, look at the discrete schemes for the described models.

Let W i  = w(x i ) so that W will be the vector solution giving the approximation of the height of u at every node x i of the grid. Following [16], the semi-Lagrangian scheme for the above models can be written in a fixed point form. In general, we will write it as

$$\displaystyle{ W_{i} = T_{i}^{M}(W), }$$
(2.39)

where M is the acronym identifying the model, then M = L, ON or PH. Denoting by G the global number of nodes in the grid, the operator for the L–model \(T^{L}: \mathbb{R}^{G} \rightarrow \mathbb{R}^{G}\) is defined componentwise by

$$\displaystyle{ T_{i}^{L}(W):=\min \limits _{ a\in \partial B_{3}}\{e^{-\mu h}w(x_{ i} + hb^{L}(x_{ i},a)) -\tau \frac{I(x_{i})a_{3}} {\omega _{3}} (1 -\mu w(x_{i}))\}+\tau, }$$
(2.40)

where τ: = (1 − e −μ h)∕μ and w(x i + hb L(x i , a)) is obtained interpolating on W.

It has been shown in [17] that the corresponding operator T L has three important properties: it is monotone, is a contraction mapping in [0, 1∕μ)G and \(0 \leq W \leq \frac{1} {\mu }\) implies \(0 \leq T(W) \leq \frac{1} {\mu }\).

Similarly, the SL fully discrete scheme for the ON–model at a node x i will be given by the discrete operator

$$\displaystyle{ T_{i}^{ON}(W):=\min \limits _{ a\in \partial B_{3}}\{e^{-\mu h}w(x_{ i} + hb^{ON}(x_{ i},a)) -\tau \frac{c(x_{i},z)a_{3}} {A\omega _{3}} (1 -\mu w(x_{i}))\} +\tau. }$$
(2.41)

The SL fully discrete scheme for the PH–model at a node x i is given by the discrete operator T PH defined as

$$\displaystyle{ T_{i}^{PH}(W):=\min \limits _{ a\in \partial B_{3}}\{e^{-\mu h}w(x_{ i} + hb^{PH}(x_{ i},a)) -\tau \frac{c(x_{i})a_{3}} {Q^{PH}(x_{i},z)}(1 -\mu w(x_{i}))\}+\tau, }$$
(2.42)

with \(Q^{PH}(x_{i},z):= 2k_{S}\gamma _{S}(d(x_{i},z)\cdot \boldsymbol{\omega }) + k_{D}\gamma _{D}\omega _{3}\).

Although the operators T ON and T PH present some differences and additional terms, they converge and have similar properties of the operator T L (see [50, 52] for the analytical proof of these properties).

In the numerical tests we will also compare results obtained with Fast Sweeping (FS) and Fast Marching (FM) methods so we briefly sketch here their properties.

3.1 Fast Marching [21, 44, 55]

For the implementation of Fast-Marching algorithm, the grid defined on the image is divided into three sets at every iteration n:

  • the set of Accepted nodes Accepted (n), whose value has been already computed and accepted;

  • the set of Considered nodes Considered (n), or Narrow Band, for which the value has to be computed at the present iteration;

  • the set of Far nodes Far (n), that are the nodes which will be computed in future iterations.

The engine of the method is the local fixed point operator. Accepted (0) at the first iteration is the set of nodes where we have to apply boundary conditions (which are known). Then, at iteration n, the set Considered (n) contains the neighboring nodes of Accepted (n) and Far (n) are the remaining nodes where we do nothing at that iteration. The algorithm computes the value in Considered (n). The node x j where the minimum is achieved is marked Accepted (i.e. Accepted (n + 1)=Accepted (n) ∪{ x j }), the set Considered (n) is updated adding the neighboring nodes to x j and we compute the solution in Considered (n + 1). The algorithms accepts only one node for each iteration and ends only when the Far region is empty. The method converges in a finite number of iterations and has a complexity of O(Gln(G)) where G is the cardinality of the grid nodes. Unfortunately, its application is limited to eikonal type equations.

3.2 Fast Sweeping [14, 40, 54]

FS is another popular method for solving HJ equations. The main advantage of this method is its implementation, which is extremely easy (easier than that of Fast Marching). FS method is basically the classical iterative (fixed-point) method, since each node is visited in a predefined order, until convergence is reached. Here, the visiting directions (sweeps) are alternated in order to follow all the possible characteristic directions, trying to exploit causality. In two-dimensional problems, the grid is visited sweeping in four directions: S → N & W → E, S → N & E → W, N → S & E → W and N → S & W → E.

The key point is the Gauss-Seidel-like update of grid nodes, which allows one to compute a relevant part of the grid nodes in only one sweep. Indeed, it is well known that in the case of eikonal equations FS converges in only four sweeps.

4 Numerical Experiments

We call G the discrete grid of points x ij , with size card(G) = n × m. We define G in : = { x ij : x ij  ∈ Ω} as the set of grid points inside Ω; G out : = G ∖ G in . The boundary ∂ Ω is defined as the set G b  ⊂ G out such that at least one of the neighboring points belongs to G in . For each image we define a map, called mask or silhouette, where the pixels x ij  ∈ G in are white and the pixel x ij  ∈ G out are black. In this way it is easy to distinguish the nodes that we have to use for the reconstruction (the nodes inside Ω) and the nodes on the boundary ∂ Ω (see e.g. Fig. 2.4b, d).

In all our numerical experiments, we neglect the ambient component that we consider as a constant (i.e. setting for simplicity k A  = 0). Our work is mainly focusing on the semi-Lagrangian approach and also our intent is to analyze the behavior of the parameters involved in the two non-Lambertian models. For these reasons, the simulations focus the attention on SL performances and on the behavior of the parameters.

4.1 Synthetic Images

The synthetic tests are useful for a quantitative analysis on the behavior of the parameters and also because it is possible to compute the error on the surface. The synthetic image that we are going to present here is defined on the domain G, that is a rectangle containing the support of the image Ω, G ≡ [−1, 1] × [−1, 1]. We can easily modify the number of the pixels choosing different values for the steps in space Δ x and Δ y. In this case we will use 256 × 256 pixels. X and Y represent the real size (e.g. for G ≡ [−1, 1] × [−1, 1], X = 2, Y = 2). As already said in Sect. 2.2, we can use homogeneous Dirichlet boundary condition but it is possible to define, if useful, the function g(x), that is the height of the surface at the boundary of the silhouette. In this test we will use this general boundary condition that we can easily derive being the object a solid of rotation. In fact, if we denote by c: = (c x , c y ) the center of the circle with ray R at the bottom of the vase we can write

$$\displaystyle{ (x - c_{x})^{2} + (y - c_{ y})^{2} = R^{2} }$$
(2.43)

and then

$$\displaystyle{ y = \sqrt{R^{2 } - (x - c_{x } )^{2}} + c_{y} }$$
(2.44)

that gives us the values of g(x).

In iterative methods, the method stops when we have reached the required tolerance η or when we have exceeded the maximum number of iterations allowed. In an iterative method of fixed point, the point is reached when | | W n+1 − W n | | < η.

4.1.1 Synthetic Vase

We use this test in order to analyze and compare the performances of the SL scheme with respect to the three different models by varying the values of the parameters involved.

The synthetic vase is defined as

$$\displaystyle{ \left \{\begin{array}{@{}l@{\quad }l@{}} u(x,y) = \sqrt{P(\bar{y})^{2 } - x^{2}}\quad &(x,y) \in G_{in}, \\ u(x,y) = g(x,y) \quad &(x,y) \in G_{out}, \end{array} \right. }$$
(2.45)

where \(\bar{y} = y/Y\),

$$\displaystyle{P(\bar{y}) = (-10.8\,\bar{y}^{6} + 7.2\,\bar{y}^{5} + 6.6\,\bar{y}^{4} - 3.8\,\bar{y}^{3} - 1.375\,\bar{y}^{2} + 0.5\,\bar{y} + 0.25)\,X}$$

and

$$\displaystyle{G_{in} =\{ (x,y)\vert P(\bar{y})^{2}> x^{2}\}.}$$

The input images generated by L–model, ON–model and PH–model with a vertical light source (\(\boldsymbol{\omega }= (0,0,1)\)) are visible in Fig. 2.2.

Fig. 2.2
figure 2

Input vase images by L–model, ON–model (σ = 0. 6), PH–model (k S  = 0. 3) with \(\boldsymbol{\omega }= (0,0,1)\)

Full size image

We show in Table 2.1 the values of the parameters related to some numerical tests performed. It is possible to compute the error in L 1, L 2, L ∞ norm on the image (L p(I)) and on the surface (L p(S)) because for synthetic images we know the real surface (for the vase this is given by (2.45)). Given a vector T representing the exact solution (or the original image) on the grid and a vector \(\tilde{\mathbf{T}}\) representing its approximation, we define the error vector as \(\mathbf{e} = \mathbf{T} -\tilde{\mathbf{T}}\) and

$$\displaystyle\begin{array}{rcl} & err_{1} = \vert \vert \mathbf{e}\vert \vert _{L^{1}} & = \frac{1} {N}\sum _{i}\vert e_{i}\vert {}\\ & err_{2} = \vert \vert \mathbf{e}\vert \vert _{L^{2}} & = \left \{ \frac{1} {N}\sum _{i}\vert e_{i}\vert ^{2}\right \}^{1/2} {}\\ & err_{\infty } = \vert \vert \mathbf{e}\vert \vert _{L^{\infty }}& = \max _{i}\left \{\vert e_{i}\vert \right \} {}\\ \end{array}$$

where N is the total number of grid points used for the computation, i.e. the grid points belonging to G in .

Table 2.1 Synthetic vase: parameter values used in the models. When a parameter doesn’t exist for a model we put a dash
Full size table
Fig. 2.3
figure 3

Synthetic vase: output images and 3D reconstructions for the three models

Full size image

The reconstructions of the surfaces and the output images obtained with the three models, starting from the input images in Fig. 2.2, are visible in Fig. 2.3.

In Table 2.2 we can observe the performances of the SL–scheme. In details, we reported the number of iterations, the CPU time (in seconds) and the error estimates in three different norms between the input image and the image reconstructed from the u approximation. Note that to obtain the reconstructed image we need an approximation of the gradient of u which is obtained via a centered finite difference which guarantees a second order accuracy. Looking at these errors, we note that the ON–model performs better increasing the parameter σ both on the image I and on the surface, with the same error order than the L–model but always lower. Instead, for the PH–model we can see that the errors on the surface decrease increasing the parameter k S , except for the case completely specular (k S  = 1), but it is not true with respect to the errors on the image that increase for increasing values of k S .

Table 2.2 Synthetic vase: numerical results for \(\boldsymbol{\omega }= (0,0,1)\) with the errors on the image and on the surface
Full size table
Table 2.3 Synthetic vase: errors on the surface via ON–model changing the size of the input image with vertical light source \(\boldsymbol{\omega }= (0,0,1)\)
Full size table
Table 2.4 Synthetic vase: errors on the surface via PH–model changing the size of the input image with vertical light source \(\boldsymbol{\omega }= (0,0,1)\)
Full size table

The errors on the images in different norms show how well the reprojection fits the input data. The L ∞ errors on the image I, that indicate the maximum over all the pixels of the difference in absolute value between the input image and the image computed as said before, are so big because if in only one point the reconstruction is not good, e.g. in a point on the boundary of the domain, then the error will be so big.

In order to confirm that the non-Lambertian models converge to the surface depth, we reported in Tables 2.3 and 2.4 the errors on the surface with respect to different size of the vase image, from 64 × 64 to 1024 × 1024 obtained doubling the size. What we can note is that increasing the number of the pixels, hence considering a smaller and smaller space step, the errors decrease for both the models.

4.2 Real Images

In this section we will consider two real input images: the bust of Beethoven (size (256 × 256)) and the black horse (size (184 × 256)), both visible in Fig. 2.4a, c.

Fig. 2.4
figure 4

Real input images and masks. (a ) Beethoven input. (b ) Beethoven mask. (c ) Horse input. (d ) Horse mask

Full size image

Unless otherwise specified, the value of η for the stopping criterion of the iterative method is fixed to 10−8 and the maximum number of allowed iterations is 9000. If a scheme arrives to the maximum of 9000 iterations, we put a ∗ before it in the table.

Obviously, for the real tests we do not know the real depth hence we cannot compute the error on the surface. The only quantity that is available is the image I, our data, and we added Tables in order to give a quantitative support to the qualitative analysis visible from the Figures reported below in the paper.

4.2.1 Beethoven

In this case, we have compared the performances of the SL–scheme applied to the three models using the parameters reported in Table 2.5 with two different cases for the light source: the vertical case (\(\boldsymbol{\omega }_{vert} = (0,0,1)\)) and the oblique case (\(\boldsymbol{\omega }_{obl} = (0.0168,0.198,0.9801)\)). This test will show better the crucial role of the parameters involved for the convergence.

Table 2.5 Beethoven: parameter values used in the models. When a parameter doesn’t exist for a model we put a dash
Full size table
Table 2.6 Beethoven: numerical results for \(\boldsymbol{\omega }_{vert} = (0,0,1)\) with the errors on the image
Full size table

As we can see in Table 2.6, in the vertical case all the models converge in less than 3 s with the same order of iteration. Looking at the errors on the images, they are of the same order for all the cases, L ∞(I) is a little bit higher for the ON–model with σ = 0. 6. We can note that in the case of σ = 0 for the ON–model and k S  = 0 for the PH–model we obtain the same errors and number of iterations too because the three models coincide as expected. With respect to the ON–model, by increasing the value of σ the errors grow slightly or remain unchanged. The same behavior has the PH–model with respect to the parameter k S . In fact, by increasing the value of k S the errors tend to increase, remaining of the same order.

Looking at Table 2.7, we can note that the oblique cases require higher CPU time with respect to the corresponding vertical cases due to the fact that the equations are more complex because of additional terms involved. Analyzing the errors on the images, as noted just before, the cases of σ = 0 for the ON–model and k S  = 0 for the PH–model coincide with the L–model in terms of number of iterations used and error estimations. With respect to the ON–model, the errors increase by increasing the parameter σ. The same holds for the PH–model with respect to k S . Because of additional terms involved in the oblique case, in Table 2.7 we have reported the results obtained using a value of the tolerance η for the stopping rule of the iterative method equal to 10−3. This is the maximum accuracy achieved by the non-Lambertian models since roundoff errors coming from several terms occur and limit the accuracy since the schemes are first order accurate. Only for the ON–model with σ = 0. 4 we have reported the result also for η = 10−4 and for the L–model with η = 10−8. Lastly, we can note that choosing k S  = 0. 4 the PH–model not converges in the maximum number of allowed iterations, i.e. in 9000 iterations.

Table 2.7 Beethoven: numerical results for \(\boldsymbol{\omega }_{obl} = (0.0168,0.198,0.9801)\) with the errors on the image
Full size table

The 3D reconstructions and the output images for the three models are visible in Fig. 2.5. The first two rows refer to the vertical case, the others to the oblique case. The reconstructions in the vertical case are more accurate than the corresponding in the oblique case also because obtained with a tolerance η = 10−3 instead of η = 10−8 as in the vertical case. Moreover, it is important to note that there is a concave/convex inversion in the reconstructed surface due to the classical ambiguity of the SfS model. This typically depends also on the correctness of the Dirichlet boundary conditions (as one can see in the synthetic vase test where we have applied a correct boundary condition u = g imposing a circular shape on that part of the boundary).

Fig. 2.5
figure 5

Beethoven: output images and 3D reconstructions for the three models

Full size image
Table 2.8 Black horse: parameters, CPU time and errors on the image with vertical light source. When a parameter doesn’t exist for a model we put a dash
Full size table

4.2.2 Black Horse

We use this test to compare the performances of the global SL–scheme with respect to the acceleration schemes (FM and FS) based on a finite difference (FD) solver (FM-FD, FS-FD). The comparison will be made for all the models (L–model, ON–model, PH–model) with the parameter values reported in the second and third column of Table 2.8. Note that the SL–scheme, that is slower than FM-FD and FS-FD methods as expected, however it is more accurate with respect to the schemes based on FD. This confirms that the SL approach is competitive with other numerical techniques. We can also note that the parameters play an important role in these models. For example, in the PH–model passing from k S  = 0. 4 to k S  = 0. 8 the errors change significantly in L 1 and L 2 norm for the FM-FD and the FS-FD methods. In Fig. 2.6 one can see the output images and the 3D reconstructions of the surface obtained by the SL–schemes applied to the three models. Note that the reconstruction obtained by the PH–model recognizes better the object in the picture and this is coherent with the fact that the surface is shiny, so the PH–model seems to be the more realistic in this case.

Fig. 2.6
figure 6

Black horse: output images and 3D reconstructions for the three models

Full size image

5 Conclusions and Perspectives

In this paper we derived the Hamilton-Jacobi equations related to three reflectance models and we presented some numerical methods to solve them. In our formulation the models share the same mathematical structure and this allows to switch on and off the different terms related to ambient, diffuse and specular reflection in a very simple way. This general model is very flexible to treat different light conditions with vertical and oblique light sources. As we noted in some of our tests, via non-Lambertian models it is not possible to solve the classical concave/convex ambiguity of the Lambertian SfS problem based on a single image despite the fact that these models can deal with more general surfaces (see [50, 52] for more details on the analysis performed on non-Lambertian models). This ambiguity can be eliminated only adding additional information on the image or dealing with more than one image as already done via the Photometric Stereo technique in the case of the Lambertian model [32, 33]. The application of the Photometric Stereo technique to models with a specular component goes beyond the scopes of this paper. Some results of this work can be found in [53].

From the numerical point of view, comparing the numerical methods we noted that the SL–schemes approximate in a rather effective way the equations related to non-Lambertian models and they are more accurate with respect to FD schemes. Looking at the numerical experiments of the three models, the PH–model seems to be the model more sensible to the values of the parameters involved, as visible in Table 2.8. This model recognizes better the object with respect to the L–model and the ON–model, although the errors computed on the image are higher. However, our numerical tests showed that all the schemes are consistent and we obtain good results for synthetic and real input images. Looking at the test performed with an oblique light source, we have some comments that are common to the PH–model and the ON–model. The equations corresponding to this case have additional terms and the corresponding discrete operators become more complex and require more iterations to converge. This produces an accumulation of floating point errors which reduces the accuracy of the approximation. Moreover, for real images, we do not know the exact direction of the light source and this introduces another perturbation in the model which affects the results. A possible improvement, at least when we know the light source direction, could be the use of second order schemes.

Another interesting direction would be to mix the models, e.g. coupling the ON–model with the PH–model. To this end, we need to verify if the new mixed model still can be written in the same fixed point form in order to apply the same approach.