GenSSS: a genetic algorithm for measured subsurface scattering representation

Abstract

We present a novel genetic algorithm-based approach for the compact representation of heterogeneous, optically thick, translucent materials. Utilizing genetic optimization, we also find the best transformation to represent measured subsurface scattering data. We employ a factored subsurface scattering representation, based on a singular value decomposition (SVD), separately applying the SVD per-color channel of the transformed profiles. In order to achieve a compact, accurate representation, we perform this iteratively on the model errors. By allowing the number of iterations to be customized, our representation provides a mechanism to trade the visual quality possible against the level of compression achieved through our representation. We validate our approach by analyzing a range of real-world translucent materials, geometries and lighting conditions. For heterogeneous translucent materials, we further demonstrate that for the same level of compression, our method achieves greater visual accuracy than alternative techniques. Finally, we present an application of our factored representation, which can be used to convert heterogeneous materials into homogeneous material representations.

1 Introduction

Translucent materials such as wax, marble and human skin allow light to partially penetrate the surface and scatter light in a more complex fashion than opaque materials. Light scattering occurs at various layers within the material, allowing some of this internal material to be seen. In computer graphics, this behavior is encoded using the general bidirectional surface scattering distribution function (BSSRDF), a generalization of the bidirectional reflectance distribution function (BRDF) introduced by Nicodemus et al. [34].

Due to the complexity of translucent materials, representing these is commonly simplified by assuming that the optical properties of the material are constant (homogeneous). In this case, light scattering can be approximated by a diffusion equation and a term for single scattering [9]. Therefore, many successful homogeneous subsurface scattering models use this approximation [6, 10, 17, 18, 21]. In practice, this is a serious limitation, as many translucent objects typically encountered are heterogeneous, i.e., their optical properties vary spatially [1, 33, 42]. Although there are a number of data-driven representations [36, 41, 42] for measured heterogeneous subsurface scattering data, efficient compression algorithms remain a challenge for representing them both compactly and accurately.

Factorization techniques have been used for some material appearance models [2, 27], bidirectional texture functions (BTFs) [46, 47], subsurface scattering (SSS) effects of homogeneous [21] and heterogeneous translucent materials [25, 36, 42]. While our novel subsurface scattering representation is also based on a factorization technique, we demonstrate, through a detailed analysis, that employing an SVD-based representation with genetic optimization [31] allows a noticeable boost to the visual accuracy of heterogeneous real-world materials for similar compression levels to alternative approaches.

We use a genetic algorithm (GA) [31] to optimize our transformation, evolving to the fittest solution giving the most accurate fit to measured subsurface scattering data. The applied transformation acts like a filtering technique, smoothing out high frequencies in the subsurface scattering profiles, which in turn demonstrably improves accuracy of the factorization.

The novel contributions of this paper are:

• A compact representation of heterogeneous optically thick translucent materials.

• A novel pre-processing technique for optimizing the transformations of profiles of measured subsurface scattering responses.

• A tunable method to trade visual complexity against compression level.

• A detailed validation of our method.

Our algorithm works by first evolving the best transformation on the profiles of measured subsurface scattering responses. After this, we separately factorize each color channel of these transformed profiles using a rank-1 approximation of the SVD-based factorization [35]. To achieve a compact and accurate representation, we iteratively apply the rank-1 approximation of the SVD to the model errors. The compactness of the rank-1 approximation also allows our GA to handle a large population in parallel. The number of iterations used is tunable. Our method achieves a subsurface scattering representation that is both optimal and the visual quality trade-off against compression level controllable (see Fig. 1).

A physically plausible BSSRDF representation should have nonnegativity, Helmholtz reciprocity and energy conservation properties [34]. A limitation of our method is that our visually plausible subsurface scattering model satisfies the nonnegativity property, the Helmholtz reciprocity, but not necessarily energy conservation.

2 Related work

Our method builds upon factorization-based representations, GAs and BSSRDF representations; therefore, we discuss each of these below.

Factorization-based representations These have been frequently used for representing BRDFs [2, 23, 27, 29, 43], spatially varying BRDFs [26], BTFs [38, 39, 46, 47], homogeneous subsurface scattering [21] and heterogeneous subsurface scattering [25, 36, 42]. We refer the reader to the tutorial by Pajarola et al.’s [35] for a survey of these techniques. Specifically, tensor approximation (TA) [24, 35] is an established factorization method that has been used in volume simulations [44, 47], BTFs [38, 39, 46, 47], BRDFs [2, 43], importance sampling of BRDFs [2] and heterogeneous subsurface scattering [25].

Nonnegative matrix factorization (NMF) is also used for representation and importance sampling of BRDFs [27] and representation of heterogeneous subsurface scattering [36, 42]. Inspired by these works, we instead employ an SVD-based factorization in combination with genetic optimization to represent measured subsurface scattering profiles. We argue that this representation gives higher visual quality for equivalent compression levels to other approaches.

Fig. 1

Framework of our subsurface scattering representation with genetic optimization

Genetic algorithms GAs have been used extensively in optimization problems, but are less frequently applied to problems in computer graphics. The most related exception is in inverse rendering to estimate the parameters of Jensen et al.’s [18] homogeneous BSSRDF model [28, 32]. This type of application makes genetic optimization very time-consuming, as the fitness function is computed between reference and rendered images for each candidate chromosome. Genetic programming (GP) together with genetic algorithms has also been used to investigate new analytic BRDF models from measured BRDF data [4]. Recently, Guarnera et al. [14] used genetic algorithms to remap the parameters of source and target analytical BRDF models. In Guarnera et al.’s [14] work, the fitness function is computed between rendered images of source and target BRDF models. In contrast to prior work, we apply genetic optimization in the accurate and compact representation of measured subsurface scattering data. Our GA is used to find the fittest transformation to profiles of measured subsurface scattering profiles. When compared to inverse-rendering-based techniques [14, 28, 32], our GA is computationally efficient, as our fitness function is computed between measured and factored data for each candidate chromosome.

BSSRDF representations A seminal paper by Jensen et al. [18] introduced the diffusion dipole approximation for homogeneous subsurface scattering in computer graphics. Subsequently, several approaches [6,7,8, 10, 16, 19,20,21, 30, 48, 49] have extended and optimized this for translucent materials and/or for real-time rendering of translucent objects.

For example, Donner and Jensen [8] extended the diffusion dipole approximation to the diffusion multipole approximation by including additional light sources into the BSSRDF computation. Mertens et al. [30] implemented the diffusion dipole approximation in screen-space to render homogeneous translucent materials in real time. d’Eon et al. [7] extended the diffusion multipole approximation [8] by approximating it with a sum of Gaussians, and used these to blur the irradiance signal in texture space. Since Gaussians allow transformation of the expensive 2D convolutions into a cheaper set of 1D convolutions, this technique can be used in real-time rendering simulations. Xu et al. [48] extended pre-computed radiance transfer (PRT) techniques to Jensen et al.’s BSSRDF model for real-time rendering homogeneous translucent materials under fixed environment lighting and real-time editing material parameters. Jensen et al.’s BSSRDF representation was extended by Jimenez et al. [19] to represent human skin in real time. Jimenez et al.’s BSSRDF representation is based on blurring the diffuse BSSRDF of the translucent object, as a post-processing step, by employing a sum-of-Gaussians formulation. Since Jensen et al.’s BSSRDF model can only be used to represent isotropic homogeneous translucent materials, Jakob et al. [16] extended it to a new anisotropic dipole approximation model for representing anisotropic homogeneous media. d’Eon and Irving’s [6] quantized diffusion approximation model is another extension of Jensen et al.’s diffusion dipole approximation model and mainly based on convolving irradiance with a sum-of-Gaussians BSSRDF approximation. This representation also uses a quantization of the Green’s function of the diffusion equation to obtain more realistic all-frequency results. Yan et al. [49] extended the quantized diffusion approximation model [6] for accurate rendering of homogeneous translucent materials under spherical Gaussian (SG) lights. Unlike previous representations [6, 18], the proposed technique can produce elliptical reflectance profiles, as it considers oblique lighting angles. Frisvad et al. [10] extended the diffusion dipole approximation by taking the direction of incoming light into account. Frisvad et al.’s analytical directional dipole model is especially suited for highly forward scattering materials, where the directionality of the incident illumination is more important. Jimenez et al. [21] used a rank-1 approximation of homogeneous subsurface scattering profiles by using the SVD technique in screen-space. Their subsurface scattering representation assumes that the irradiance is additively separable in order to render homogeneous translucent materials at real time. However, none of these representations have been designed to represent heterogeneous translucent materials accurately and compactly.

Fuchs et al. [11] used a linear combination of exponential fall-off functions to represent heterogeneous translucent materials, which were measured with a DISCO laser-based acquisition system [12]. For quasi-homogeneous translucent materials, i.e., those with uniformly distributed heterogeneous structures, Tong et al. [45] proposed a quasi-homogeneous representation. Peers et al. [36] employed a combination of k-means clustering and NMF-based algorithms to represent measured heterogeneous subsurface scattering data. First, Peers et al. represented the homogeneous part of a heterogeneous translucent material using k-means clustering. Next, they factorized the heterogeneous part after division by a homogeneous approximation of the heterogeneous translucent material using an NMF-based algorithm. Song et al.’s [41] SubEdit representation allows interactive editing and rendering of heterogeneous translucent materials, trading-off efficiency in compactness and accuracy. The SubEdit representation uses the logarithm of measured BSSRDF data and decomposes the logarithmically transformed BSSRDF into a set of 1D scattering profiles defined at each point. This decoupling enables editing operations while maintaining the heterogeneity present in the measured data [41]. Chen et al. [5] implemented Song et al.’s SubEdit representation in screen-space for real-time rendering of deformable, heterogeneous translucent objects. Kurt et al. [25] used the Tucker factorization technique [2] with a linear regression to represent measured heterogeneous subsurface scattering data. In Kurt et al.’s work, the Tucker factorization was used to represent intensity of measured subsurface scattering responses. Then, to capture color values, linear regression was separately applied to each color channel. Kurt et al.’s [25] representation gives rise to results similar to Peers et al.’s [36] with respect to model accuracy and results similar to ours in terms of compression. More recently, Sone et al. [40] and Nakamoto and Koike [33] proposed a parameter estimation method, using existing homogeneous BSSRDF models [6, 10, 18] for rendering heterogeneous translucent materials. This method estimates optical properties of homogeneous volumes by taking the average of coefficients of analytical BSSRDF models around the incident and exitant points. These average coefficients allow representation of heterogeneous translucent materials efficiently by using homogeneous BSSRDF models.

In contrast to the above methods, our approach uses a genetic optimization to discover the fittest transformation to the measured subsurface scattering responses. Unlike prior work, our GA combined with the SVD-based technique provides a new approach for representing measured subsurface scattering data of translucent materials.

3 Overview

The general behavior of translucent materials described by the BSSRDF [34] $S(x_i,\overrightarrow{\omega }_i;x_o,\overrightarrow{\omega }_o)$ relates an outgoing radiance $L_o(x_o,\overrightarrow{\omega }_o)$ at a location $x_o$ in an outgoing direction $\overrightarrow{\omega }_o$, to an incoming radiance $L_i(x_i,\overrightarrow{\omega }_i)$ at a location $x_i$ in an incoming direction $\overrightarrow{\omega }_i$ as follows:

$$\begin{aligned}&L_o(x_o,\overrightarrow{\omega }_o) \nonumber \\&= \int _A\int _{\Omega ^+} L_i(x_i,\overrightarrow{\omega }_i)S(x_i,\overrightarrow{\omega }_i;x_o,\overrightarrow{\omega }_o)(\overrightarrow{\omega }_i \cdot \overrightarrow{n})d\overrightarrow{\omega }_i\mathrm{d}x_i,\nonumber \\ \end{aligned}$$

(1)

A is the area around the location $x_o$, $\Omega ^+$ is the hemisphere around the location $x_i$, and $\overrightarrow{n}$ is the surface normal at the location $x_i$. Equation (1) can be seen as an extended version of the rendering equation [18, 22], and it can be separated into two components: a local component $L_l(x_o,\overrightarrow{\omega }_o)$ and a global component $L_g(x_o,\overrightarrow{\omega }_o)$. While the local component represents light immediately reflected from a surface, the global component accounts for the light scattering within the material volume. As in Goesele et al. [12], Peers et al. [36] and Song et al. [41], we represent the global component by the diffuse BSSRDF $S_d(x_i,\overrightarrow{\omega }_i;x_o,\overrightarrow{\omega }_o)$ which can be further decomposed approximately [8, 18]:

$$\begin{aligned} S_d(x_i,\overrightarrow{\omega }_i;x_o,\overrightarrow{\omega }_o)= & {} \frac{1}{\pi }F_i(x_i,\overrightarrow{\omega }_i)R_d(x_i,x_o) \nonumber \\&F_o(x_o,\overrightarrow{\omega }_o), \end{aligned}$$

(2)

$R_d(x_i,x_o)$ is a four-dimensional ($4D$) spatial subsurface scattering component, $F_o(x_o,\overrightarrow{\omega }_o)$ and $F_i(x_i,\overrightarrow{\omega }_i)$ are directionally dependent components. In contrast to previous works [12, 25, 36, 41], we apply genetic optimization together with SVD for accurate, compact and efficient representation of the $4D$ spatial component $R_d(x_i,x_o)$ of translucent materials. Similarly, we ignore the local component $L_l(x_o,\overrightarrow{\omega }_o)$ and the directional dependent components $F_o(x_o,\overrightarrow{\omega }_o)$ and $F_i(x_i,\overrightarrow{\omega }_i)$ by assuming light incoming from the surface normal.

4 Our subsurface scattering representation

In this section, we describe the key steps in our framework for deriving our measured subsurface scattering representation (see Fig. 1), together with our genetic algorithm. Through the use of our GA, we discover transformations that give rise to the fittest representation. Finally, we analyze factorization-based representations, and the impact of the most important parameters of our factored subsurface scattering representation.

4.1 Our genetic algorithm

Genetic algorithms are machine learning approaches used to solve both constrained and unconstrained optimization problems. They are loosely based on natural selection [31]. We apply a GA in two ways, first to discover new transformations for real-world subsurface scattering and second to represent measured subsurface scattering data using an SVD-based technique.

For our transformation search problem, we evolve a population to find the fittest solution. Each candidate solution is referred to as a chromosome consisting of estimated transformation parameters (called genes). The set of chromosomes that make up the population is initialized randomly for the first generation. Through selection, crossover and mutation operations, the population is evolved over successive generations with the help of a fitness function. To discover new transformations for real-world subsurface scattering, we can use a simple nonlinear optimization, which would generate a single point at each iteration. The sequence of points would approach an optimal solution with the algorithm deterministically selecting the next point in the sequence. However, in nonlinear space, global search problems are challenging and it is harder to robustly find global minima. Nonlinear optimization pursues a single solution to the search problem. GAs, on the other hand, try several potential solutions (i.e., population) to global search problems. Therefore, GAs converge to a more accurate result. For this reason, we opt instead to also use genetic optimization for this part of our technique.

Our fitness (i.e., loss function) of a chromosome is defined as the root-mean-square error (RMSE) between measured and factored subsurface scattering data. The population size (P) is equal to $10\times N$, where N is the number of genes in the chromosome. In other words, N is equal to the number of variables in the applied transformation. In the evolutionary process, some of the chromosomes are considered as elite, which gives lower fitness values in the population. The fraction of elite chromosomes is equal to $\lceil 0.05\times P \rceil$, which specifies the number of chromosomes guaranteed to survive through to the next generation.

A selection function specifies the choice of parents for the next generation. This function can be thought of as a stochastic uniform, which arranges a line in which each parent corresponds to a section of the line of length proportional to its scaled value. These values are determined by scaling the raw scores (i.e., RMSE) based on the rank of each chromosome. The rank (r) of a chromosome is its position (proportional to $1/\sqrt{r}$) in the sorted raw scores. Using the square root in the rank fitness scaling makes poorly ranked chromosomes more nearly equal in score giving them a chance to survive. The algorithm traverses along the line in steps of equal size. At each step, a parent is allocated from the section it lands on. The first step is selected by a uniform random number less than the step size (see Goldberg [13] for details).

A crossover function can be called as the scattered function since it produces a random binary vector [13]. The genes from the first parent are selected when the vector is 1, and the genes from the second parent are selected when the vector is 0. These are combined to specify the genes for the child. Our crossover fraction is equal to $0.8 \times P$, which specifies the fraction of the next generation, other than elite children, that are produced by the crossover operation. The mutation function randomly generates directions that are adaptive with respect to the last successful or unsuccessful generation. This function chooses a direction and step length that satisfies bounds and linear constraints [13]. Our mutation fraction is approximately equal to $0.2 \times P$, which specifies the fraction of the next generation, other than elite children, that are produced by the operation.

Fig. 2

a, b First, we reformat the BSSRDF matrix (a) by aligning the diagonal by a change of variables to $R'_d(x_i, d)$, similar to [36]. b, c Second, we automatically apply various transformation operations (see Table 1) to obtain the subsurface scattering matrix $R''_d(x_i, d)$ by using our genetic algorithm. To reconstruct $R'_d(x_i, d)$ from $R''_d(x_i, d)$, we only need to store the fittest parameters (i.e., the fittest chromosome) of the applied transformations.

Table 1 Properties of the genetic optimization for heterogeneous artificial stone

To achieve meaningful and visually plausible results, we apply boundary constraints to genes in the applied transformation (see Sect. 4.4 for the applied boundary constraints). Our genetic optimization is terminated when the average relative change in the best fitness function value, over generations, is less than or equal to a small threshold value (i.e., $10^{-6}$). In our GA, we always use the same values and fractions and users do not need to tune many parameters.

Fig. 3

From left to right: red, green and blue channels of measured scattering profiles ($R'_d(x_i, d)$) for artificial stone, when $x_i = 0$. Measured scattering profiles have their peaks when $d = x_o - x_i = 0$, which is useful information for finding the fittest transformation with our genetic algorithm. Note that for better visualization, measured scattering profiles were applied a logarithmic-based scaling. Note also that d, $x_o$ and $x_i$ are all scalar values

4.2 Transformation

Measured BSSRDF data can be represented as a matrix $R_d(x_i, x_o)$ where $x_i$ and $x_o$ are incoming and outgoing surface locations. In our transformation framework, similar to Peers et al. [36], we first linearize the measured $4D$ input data to a two-dimensional ($2D$) form, i.e., matrix. Figure 2a shows an example of a subsurface scattering matrix $R_d(x_i,x_o)$ of a heterogeneous material containing a light blocking element, for example a vein. The effects of this are expressed by horizontal and vertical discontinuities in $R_d(x_i,x_o)$. Second, we reorganize the subsurface scattering matrix $R_d(x_i,x_o)$ by changing the variables $d = x_o - x_i$ (see Fig. 2a, b). The reparameterized subsurface scattering matrix $R_{d}'(x_i,d)$ is a more compact form of the measured BSSRDF data (since $\max (d) = kernel size$, $\max (d) \ll \max (x_o)$, and $R_{d}'(x_i,d) \approx 0$ when $x_o - x_i > \max (d)$), and it can be factorized instead of $R_d(x_i,x_o)$.

To arrive at the fittest transformation for the measured subsurface scattering data, we apply various transformations using our GA (described in Sect. 4.1, see Figs. 1, 2b, c), to measured data sets from Peers et al. [36] and Song et al. [41] by using the following steps:

• 1. We automatically generate transformations from initial seed transformations.

• 2. We automatically generate a range of chromosomes for every applied transformation.

• 3. By using our genetic optimization framework, we apply the transformations with a rank-1 approximation of SVD to subsurface scattering data to find the fittest transformation.

Table 1 shows these applied transformations and their properties. We seed the search with some handpicked (empirically derived) initial transformations (see transformation ID: $1 \& 4 \& 11$ in Table 1); we then automatically optimize these by trying variations of the initial transformations that are close by in the search space. Since subsurface scattering profiles are large data sets, we handle a significant amount of data (i.e., GBs of data). These large data sets increase the computation time of our GA, and naïvely representing a material may take about $1-2$ days. In order to make this more tractable, we perform this as a pre-processing step and we automatically try a range of chromosomes for the same transformation reducing computation times to $\sim 2-3 \times$. This allows us to optimize chromosomes for the applied transformation. Our GA finds lower fitness values if transformations are applied to each color channel separately (see transformation ID: $1 \& 2$ in Table 1). This is because the measured subsurface scattering data are the diffuse reflectance component which differs in all color channels. In transformations ID:10 through 15, $R_{d}'(x_i,d)$ is divided by its maximum value (i.e., $\max (R_{d}'(x_i,d))$). Converting the subsurface scattering profiles in this way ensures that the values of subsurface surface scattering profiles remain within [0, 1]. This greatly improves the efficiency of our GA for finding the fittest transformation with the least generations (see transformation ID:$9 \& 13$ in Table 1). This is an important observation, as the number of generations plays a key role in finding the fittest transformation, and increases the optimization time of GAs linearly [28, 32].

Measured subsurface scattering profiles have their peaks when $d = x_o - x_i = 0$ (i.e., the light and the viewer are at the same position), and other parts of the subsurface scattering profiles are primarily diffuse as can be seen in Fig. 3. Starting from $d=0$, we allow our GA to find the fittest range of the applied transformation. Discovering this range helps to find the most optimal transformations, these were found to be transformations ID: $14 \& 15$ shown in Table 1. Discovering this range also helps to optimize the computation time of our subsurface scattering representation, as we do not need to apply the transformation to each surface point. We applied both transformation ID: 14 and 15 to all measured subsurface scattering data with a rank-1 approximation of SVD shown in Fig. 4. On average, transformation ID: 14 and 15 decrease the RMSE by $3.835\%$, and $2.875\%$, respectively. Therefore, we select and use transformation ID: 14 in the rest of paper. We get more erroneous representations with lower rank approximations of SVD. Our GA together with the applied transformation allows us to achieve visually plausible results even with lower rank approximations of SVD.

Fig. 4

A comparison of the SVD-based subsurface scattering model with and without applying various transformations (see Table 1) using our genetic optimization. The model parameter K was selected as 1. The error values were sorted in the logarithmic RMSEs of the SVD technique (purple) for visualization purposes

Fig. 5

The rank-k approximation of the measured subsurface scattering matrix $R''_d(x_i, d)$ is composed of matrix U, matrix S and matrix V. We use the rank-1 approximation of the measured subsurface scattering matrix $R''_d(x_i, d)$

Fig. 6

We use an error modeling approach [2] using SVD-based factorization for representing the measured subsurface scattering matrix $R''_d(x_i, d)$

4.3 Factorization

We compare representing the measured subsurface scattering data with the following techniques:

• TD (Tucker-based factorization [25]).

• NMF-based technique, similar to Peers et al. [36].

• SVD [35].

Fig. 7

A comparison of our SVD-based subsurface scattering representation and a Tucker-based subsurface scattering representation [25] at nearly equal storage levels

Fig. 8

A comparison of our SVD-based subsurface scattering representation and a NMF-based subsurface scattering representation [36] at equal storage levels

The SVD technique gives the best low-rank approximation with respect to the Frobenius norm on 2D data [21, 35], and it is also the most computationally efficient approach on 2D data. The SVD computes the k largest singular values and associated singular vectors of matrix $R''_d(x_i, d)$; thus, the SVD produces $x_i \times k$-dimensional matrix U, $k \times k$-dimensional matrix S and $k \times d$-dimensional matrix V, which yields $R''_d(x_i, d) \approx USV$. In our modeling procedure, we use the SVD to compute the rank-1 approximation of the $R''_d(x_i, d)$ by setting $k=1$, which yields two vectors ($f_1(x_i)$ and $v_1(d)$) and a scalar value ($s_1$) in the form of $R''_d(x_i, d) \approx f_1(x_i)s_1v_1(d)$. To increase compactness and to reduce rendering time of our representation, we multiply $v_1(d)$ with $s_1$ to get $h_1(d)$, thus $R''_d(x_i, d) \approx f_1(x_i)h_1(d)$. This modeling process is depicted in Fig. 5.

Table 2 Properties of the factored heterogeneous subsurface scattering materials

Fig. 9

a The root-mean-square error (RMSE) values (lower is better) of our SVD-based model for various values of K; b the peak signal-to-noise patio (PSNR) values (higher is better) of our SVD-based model on the dragon scene (see Fig. 11) for various values of K; c the compression ratio (CR) of our model for various values of K; d the rendering times of our model on the dragon scene (see Fig. 11) for various values of K

Fig. 10

For parameter analysis of our subsurface scattering model on Lucy, Buddha, kitten and dragon objects under spot lighting, (first column) heterogeneous artificial stone, chessboard ($8 \times 8$), marble (close up) and yellow wax materials were rendered with a Monte Carlo path tracing algorithm (reference images); (other columns) the same heterogeneous translucent materials were rendered using our factored subsurface scattering model for various values of K. Below each image we also report the RMSE value (lower is better) and PSNR value (higher is better). Insets show false-color difference images computed between the reference images and the rendered images. Note that for better comparison, false-color differences were scaled by a factor of ten. All images were rendered at 16 samples/pixel

Our GA combined with the rank-1 approximation of the subsurface scattering matrix using an SVD approach provides a very compact, visually plausible and computationally efficient representation. However, due to the approximate nature, it is unsuitable for accurately representing the visual complexity of scattering behavior from highly heterogeneous and anisotropic structures of translucent materials [21]. Therefore, to minimize approximation errors, we use an error modeling approach with the rank-1 approximation of $R''_d(x_i, d)$ using the SVD technique for flexible representation. This type of error modeling approach was first introduced by Bilgili et al. [2]. Similar to their work, we simplify the approximation procedure and reduce the error of approximation in a stepwise fashion. The process for this is firstly, we decompose the $R''_d(x_i, d)$ matrix with the SVD-based factorization yielding the vectors ($f_1(x_i)$ and $h_1(d)$). Next, we decompose the model errors ($e_1 = R_d''(x_i,d)- f_1(x_i)\, h_1(d)$) again with the rank-1 approximation of the SVD-based factorization, giving the vectors ($f_2(x_i)$ and $h_2(d)$). Then, we factorize the model errors ($e_2 = R''_d(x_i,d)- f_1(x_i)h_1(d)-f_2(x_i)h_2(d)$) again with the rank-1 approximation of the SVD-based factorization, giving the vectors ($f_3(x_i)$ and $h_3(d)$). Finally, this process is repeated for a predetermined number of times. As shown in Fig. 6, the final subsurface scattering model will be the sum of the estimation of model errors and the first factorization of $R''_d(x_i, d)$, which can be formalized as:

$$\begin{aligned} R_{d}''(x_i,d) \approx \sum _{j=1}^{K}f_j(x_i)h_j(d), \end{aligned}$$

(3)

K is the total number of terms and $f_j(x_i)$ and $h_j(d)$ are the univariate functions, which help to provide a very compact subsurface scattering representation. In our subsurface scattering representation, K provides some controllability of the modeling errors of our representation. Since $f_j(x_i)$ and $h_j(d)$ functions are represented as piecewise linear functions, we apply a bilinear interpolation on d when computing $R_{d}''(x_i,d)$. This bilinear interpolation is required for the visualization step, to match our texture-space representation to 3D objects described in geometry space. Note that when $K=n$, this is the total number of observations (i.e., the number of total measurements) required for the model to provide a perfect representation.

Instead of using K times a rank-1 approximation of SVD with the error modeling approach, we could simply use a rank-k approximation of SVD to represent measured subsurface scattering profiles. Both representations would give the same RMSE, theoretically. However, the rank-1 approximation of SVD is more compatible with our GA, and since we use a parallel version of our genetic algorithm, the population size is handled in parallel requiring $3 \times (x_i \times k + k \times k + k \times d) \times P$ memory. Choosing to use the rank-1 approximation allows our GA to increase the population size, which helps to find the fittest representation more accurately.

We compare our SVD-based subsurface scattering representation (GenSSS) with other factorization-based subsurface scattering representations. To make a fair comparison, we compare these representations at equivalent storage levels. In our comparisons, we use measured data from Peers et al. [36] and Song et al. [41]. In Fig. 7, we compare our SVD-based subsurface scattering representation with the Tucker-based subsurface scattering representation [25]. The latter uses Tucker factorization on the intensity channel. To compute color values, it applies polynomial approximation to each color channel, separately. In Fig. 8, we compare our SVD-based subsurface scattering representation with the NMF-based subsurface scattering representation, similar to  [36]. The NMF-based representation applies nonnegativity constraints on the factorization of subsurface scattering profiles, and it applies the NMF technique to each color channel separately. As shown in Figs. 7 and 8, our SVD-based subsurface scattering representation (GenSSS) gives more visually accurate results than alternative factorization-based subsurface scattering representations.

4.4 Parameter analysis

Our subsurface scattering representation (GenSSS) has four parameters: $\max (R_{d}'(x_i,d))$, $\alpha _s$, range and K. The parameter $\max (R_{d}'(x_i,d))$ is computed from measured subsurface scattering profiles. $\alpha _s$ and range are other components of transformation ID:14. They are both scalar values and determined through our GA. Our algorithm searches for the fittest chromosome, consisting of $\alpha _s$ and range. To get meaningful and visually plausible results, we apply some constraints to $\alpha _s$ and range, such that $0 < \alpha _s \le \infty$, $0 \le range \le \max (d)$. K is the number of terms in the SVD-based factorization used in our model.

Fig. 11

A visual comparison between the SubEdit model [41] and our model at comparable data sizes on dragon, Buddha and statue objects under spot lighting. Note that for better comparison, false-color differences were scaled by a factor of ten. All images were rendered at 16 samples/pixel

The value of K can be configured for the usage context of our subsurface scattering model. This provides some flexibility in our subsurface scattering representation, and control over the visual quality of our subsurface scattering representation. As can be seen in Table 2, Figs. 9 and 10, when K is chosen to be 5, our model represents the general shape of the heterogeneous subsurface scattering profile accurately while providing good rendering times and compression rates. The rendering time of our representation increases linearly as the number of terms in the SVD-based factorization increases, which can be seen in Fig. 9d. Minimizing K in our model plays a key role when lower rendering times are desirable. Another important parameter that has a key role on rendering times is the range, as we apply the transformation ID:14 to the range of subsurface scattering profiles. Our GA helps us to find the optimum range of applied transformations, which also optimize the rendering times of our GenSSS model. As Fig. 10 shows, we achieve visually plausible results even with $K = 1$ for most of the translucent materials represented.

Fig. 12

A diffuse albedo comparison between Song et al.’s SubEdit model [41] and our GenSSS model at comparable data sizes. We also computed false-color difference images between measured albedos and corresponding approximations of models. Note that for better comparison, false-color differences were scaled by a factor of ten. We also report PSNR values (higher is better). Although chessboard ($4 \times 4$) and jade materials exhibit high anisotropy and interesting subsurface scattering structures, our subsurface scattering model exhibits these behaviors quite well

Fig. 13

A response comparison between the SubEdit model [41] and our model at comparable data sizes. For each material, the diffuse albedo map, a relative error distribution plot of our representation and a selection of measured responses with the corresponding models approximations are shown. The locations of the responses were marked on the diffuse albedo map. We also computed false-color difference images between measured responses and corresponding approximations of models. Note that for better comparison, false-color differences were scaled by a factor of ten. We also report PSNR values (higher is better)

5 Results

We implemented a rendering scheme similar to Peers et al. [36] using the Mitsuba rendering system [15] to visualize our results and make visual comparisons with existing subsurface scattering representations. All subsurface scattering representations are implemented in texture space, and we use classical path tracing with 4 bounces of interreflections. Our rendering algorithm includes a pre-processing part that computes the irradiance on a large set of sample positions spread uniformly over the object surface. These blue noise sample points are well distributed, and their locations are chosen using a technique proposed by Bowers et al. [3]. The blue noise samples are convolved with the diffusion subsurface scattering profiles using a fast hierarchical technique [17]. To validate our subsurface scattering model and compare it with existing subsurface scattering representations, we use measured heterogeneous subsurface scattering data sets from Peers et al. [36] and Song et al. [41]. In our measured subsurface scattering representation procedure, we use a parallel implementation of GAs [13, 31], and an implementation of the SVD function [35] in MATLAB. We perform comparisons on 8 different real-world translucent materials, varying from fairly homogeneous to highly heterogeneous materials. Table 2 gives an overview of the modeled heterogeneous translucent materials and lists a number of statistics for our GenSSS model, based on typical values for K. These results demonstrate that our representation gives a high compression ratio together with high visual accuracy.

Figure 10 shows the results from using GenSSS with varying values of K for artificial stone, chessboard ($8 \times 8$), marble (close up) and yellow wax materials. We report the root-mean-square error (RMSE), the peak signal-to-noise ratio (PSNR) [37] and false-color difference images to better illustrate the effects of the parameter K used in our representation. These results show that our method provides high compression ratios (CRs) when lower values of K are used. At the same time, our representation captures visually plausible heterogeneous subsurface scattering effects. When higher values of K are preferred, we achieve high accuracy, but at the expense of compactness and rendering times, as shown in Fig. 9.

In Fig. 11, we compare our model with Song et al.’s [41] SubEdit representation on a selection of heterogeneous translucent materials. To better illustrate the effects of subsurface scattering, the scenes are rendered under spot illumination using a Monte Carlo path tracing algorithm. All scenes are rendered at 16 samples/pixel with 4 bounces of interreflections. We set K to 10 for all translucent materials to compare both subsurface scattering representations at comparable data storage levels. We again report the RMSE, the PSNR [37] and false-color difference images to better visualize the differences between the representations. Figure 12 shows a comparison of diffuse albedos to better visualize high anisotropy and interesting subsurface scattering structures, as both jade and the chessboard ($4 \times 4$) exhibit these behaviors quite well [42]. As seen in Figs. 11 and 12, our GenSSS model captures highly anisotropic translucency effects and interesting heterogeneities more accurately for comparable data storage requirements.

Similar to Peers et al. [36], and Song et al. [41], we also compare measured and modeled subsurface responses of selected surface points in Fig. 13. In this comparison, the dashed square approximately equals the kernel size and illustrates the relative size of the responses. The data storage sizes and RMSE values of compared subsurface scattering representations are reported in Fig. 11. The SubEdit representation is designed for editing heterogeneous translucent materials. It can be viewed as a decomposition of the diffuse BSSRDF, into the product of two local scattering profiles defined at the incident and outgoing surface locations. These spatially varying 1D scattering profiles increase the storage demands for the SubEdit representation. The SubEdit representation may show radially symmetric behavior at some materials (see blue wax and yellow wax in Fig. 13), due to the parameterization used, which may be insufficient for representing heterogeneous materials accurately. The comparisons outlined show that our GenSSS model represents heterogeneous translucent materials more accurately for comparable data storage requirements.

We also compare our model with the NMF-based subsurface scattering representation, see Fig. 14. In the data fitting procedure, we minimize the RMSE values of the subsurface scattering representations and use a MATLAB implementation of the NMF technique [35]. Figure 14 may also be considered as a comparison between Peers et al.’s factored subsurface scattering model [36] and our subsurface scattering model at equal data sizes, since the core part of Peers et al.’s factored subsurface scattering model is based on the NMF technique. In Fig. 15, we also compare Peers et al.’s factored subsurface scattering model and our model at comparable data sizes. As shown in Fig. 15, our GenSSS model represents responses around the peak (i.e., $d = x_o - x_i \approx 0$) more accurately. This is mainly due to our applied transformation together with our genetic optimization. NMF-based factored representations enforce nonnegativity in the data fitting process, and this constraint causes an increase in modeling errors. Our method does not enforce nonnegativity in the data representation process; instead, we satisfy this when discovering the fittest transformation by applying constraints to the parameters in our GA. While NMF-based representations [36] decompose BSSRDF data into the products of matrices, our GenSSS representation consists of the univariate functions. Consequently, our model represents heterogeneous translucent materials more accurately than the NMF-based factored subsurface scattering representations for comparable data storage requirements.

Table 3 compares storage needs of various subsurface scattering representations. We selected a value for K as 5 for all the translucent materials used in this comparison. The results show that our model is ($\sim 2.32 \times$) more compact than the SubEdit representation [41] and ($\sim 3.71 \times$) more compact than Peers et al.’s factored subsurface scattering representation [36]. This is due to the fact that our representation needs only one-dimensional (1D) scattering profiles (9 scalar values from the transformation ID:14, K times $f_j(x_i)$ and $h_j(d)$ in Eq. 3). In contrast, Peers et al.’s [36] representation requires a higher number of terms and larger matrices and Song et al.’s [41] needs higher sizes of spatially varying 1D scattering profiles. Finally, Table 4 lists a comparison of rendering times for various subsurface scattering representations. Again, we set K to 5 for all translucent materials used in this comparison. All our computations were performed on a dual Intel Xeon E5-2640 v3 CPU@2.6 GHz with 80 GB RAM workstation. These results show that GenSSS gives comparable rendering times to Song et al.’s [41] SubEdit representation (on average, our model is $\sim 0.1$ minutes faster than the SubEdit model), which is designed for editing and near real-time rendering of heterogeneous translucent materials. The SubEdit representation uses a lower number of terms, but it applies both square root and exponential transformations to all surface points. Our model applies an exponential transformation (from transformation ID:14) to only an optimum range of response functions, found by our GA. Real-time rendering and editing of Song et al.’s [41] SubEdit representation would require an alternative rendering technique from the Monte Carlo path tracing algorithm we employ for this study. Such an alternative rendering technique would also work with our GenSSS representation. Furthermore, GenSSS is much faster than Peers et al.’s [36] representation (on average, our model is $\sim 2.2$ min faster than Peers et al.’s model), as ours requires a lower number of terms compared with Peers et al.’s approach for rendering translucent materials. Peers et al.’s model computes subsurface scattering profiles through a multiplication of matrices of homogeneous and heterogeneous approximations making it computationally intensive.

Fig. 14

A visual comparison between the NMF-based subsurface scattering model [36] and our model at equal data sizes on Buddha, statue, dragon and Lucy objects under spot lighting. Note that for better comparison, false-color differences were scaled by a factor of ten

Fig. 15

A response comparison between Peers et al.’s factored model [36] and our model at comparable data sizes. For each material, the diffuse albedo map, a relative error distribution plot of our representation and a selection of measured responses with the corresponding models approximations are shown. The locations of the responses were marked on the diffuse albedo map. We also computed false-color difference images between measured responses and corresponding approximations of models. Note that for better comparison, false-color differences were scaled by a factor of ten. We also report PSNR values (higher is better)

6 Applications

This section shows examples of how the measured heterogeneous subsurface scattering data can be converted into a homogeneous subsurface scattering representation by simply using a rank-1 approximation of SVD.

To construct the homogeneous BSSRDF data that exhibits radially symmetric behavior (locally homogeneous), we use the reparameterized matrix $R'_d(x_i, d)$, and apply the following computation, similar to Peers et al. [36]:

$$\begin{aligned} G_{d}(x_i,x_o) = g(d) = g(x_o - x_i) = {avg}_{x_i}{(R_{d}'(x_i,d))}, \end{aligned}$$

(4)

where g(d) is an average response function. $G_{d}(x_i,x_o)$ is an approximation of the homogeneous BSSRDF data. Therefore, g(d) can be used to create homogeneous BSSRDF data $G_{d}(x_i,x_o)$. After we compute the homogeneous BSSRDF data $G_{d}(x_i,x_o)$ from $R_{d}(x_i,x_o)$, we use the same reparameterization in Fig. 2 to compute $G_{d}'(x_i,d)$. The computation in Eq. 4 is essentially a radial averaging of the BSSRDF profiles. After we construct the homogeneous BSSRDF data using Eq. (4), we represent it with a rank-1 approximation of SVD. Figure 16 shows that our homogeneous subsurface scattering representation provides an almost ideal representation (RMSE $=$ 0 for all materials), as $h_1(d)$ in Eq. 3 mostly equals g(d) in Eq. 4. Our representation is easy to compute ($K =1$) and provides a very compact solution, since we simply store g(d) for each color channel separately. It can also be used in real-time rendering of homogeneous translucent materials.

Table 3 Required storage spaces of various subsurface scattering representations for different heterogeneous translucent materials

Table 4 Rendering times (in minutes) of various heterogeneous subsurface scattering representations on a dragon scene under spot lighting with a Monte Carlo path tracing algorithm (see Fig. 11)

Fig. 16

By using our subsurface scattering model, homogeneous subsurface scattering effects can be captured from measured translucent materials

7 Conclusions and future work

In this paper, we introduce our GenSSS representation for measured subsurface scattering data. This is based on a genetic algorithm together with the SVD technique. We describe our genetic optimization framework, geared toward discovering the fittest transformation for measured subsurface scattering data. The accuracy, efficiency and compactness of our model are validated through comparisons with a selection of different real-world translucent materials. We show that our compact GenSSS representation can be applied to any geometry and can be easily integrated into a standard global illumination rendering system while yielding convincing images.

Furthermore, we present an analysis of the effects of parameters of our subsurface scattering model on accuracy, compactness and efficiency of our GenSSS model. We also compare our approach with Peers et al.’s [36] factored and Song et al.’s [41] SubEdit models. Through this comparison, we demonstrate that our GenSSS model can represent heterogeneous subsurface scattering effects more compactly and accurately than previous methods, also offering good rendering times. This makes our representation a suitable candidate for real-time rendering applications. Finally, we demonstrate that our subsurface scattering representation can be used to convert heterogeneous translucent materials into homogeneous ones. In the future, we plan to investigate rendering algorithms for employing our GenSSS representation in screen-space, to render translucent materials in real-time applications. We are also interested in exploring the use of general solutions and reparameterizations for even better representations of homogeneous, quasi-homogeneous and heterogeneous subsurface scattering profiles utilizing our genetic optimization framework.