Keywords

1 Introduction

When designing near-body products, product developers often rely on virtual design mannequins. The disadvantage of current mannequins is that they are a simplified representation of the population [1]. Statistical shape modeling is a well-known technique in 3D anthropometric analyses. It allows gaining a better understanding of the variation in shape present in a population. SSMs are highly valuable for product designers since ergonomic products for a specific target population can be designed from these models. By adapting the shape parameters of the SSM, a new, realistic shape can be formed. Product developers may exploit SSMs to serve as input for virtual design mannequins and explore the body shapes belonging to a specific percentile of a target group, to visualize extreme shapes. On the other hand, an SSM can visualize a specific body shape, which is useful for customization. Reed et al. [2] developed a pose modification technique. Nevertheless, posture has a high influence on their results.

When building an SSM from 3D body scans, body posture has a significant influence on the shape modes. Even when the subjects are instructed to maintain a standard pose, there are slight posture variations noticeable, especially in the region of the arms. This is the case in e.g. the Civilian American and European Surface Anthropometry Resource (CAESAR) database [3]. As a result, some shape variances are incorrectly correlated with posture.

Most posture deformation techniques are based on rotating and translating the body parts. In particular, applying linear blend skinning (LBS) to a surface mesh with a simplified skeleton consisting of joints and bones, is a popular approach [4]. Posture normalization is commonly done by transforming each body part separately. The disadvantage of LBS, is that the result may look unnatural, because LBS cannot compensate for muscle bulging.

Another common approach is separating the shape space from the posture space. Wuhrer et al. [5] did a posture-invariant shape analysis using the Laplace operator. This is very computationally intensive for surface meshes with many vertices, because an optimization problem needs to be solved for every vertex. The SCAPE method [6] is a data-driven method that learns a posture deformation model that derives the non-rigid shape deformation as a function of the posture of the articulated skeleton. The advantage of this technique is that it generates more realistic shapes including the simulation of muscle bulging. However, a drawback of SCAPE is that a skeleton has to be defined for every surface mesh, which is time-consuming and error-prone.

In this paper, we propose a fast, skeleton-less, data-driven method to perform statistical shape modeling in a posture invariant way by minimizing the influence of posture, because we solely want to capture body shape variability. The remainder of this paper is organized as follows. First, the approach for building an SSM is detailed. Second, the approach for feature modification and identity removal is described. Then, the technique for posture normalization is discussed. In the results section, the several components of the framework are tested and evaluated. Finally, a conclusion is formulated.

2 Methods

In this section, the developed framework is described. First, an SSM is built from a population of 3D human body shapes [7]. Next, the identity of these training shapes is removed, by modifying the features of each shape so that they are equal to the average features. From these shapes without identity, a posture model is built. This model captures only the posture of a new input shape. The translation of each vertex from the posture model to the mean posture is calculated and applied to the input shape, which results in a posture-normalized shape.

2.1 Statistical Shape Model (SSM)

An SSM is built from a population of \( N \) shapes, with every shape consisting of \( n \) vertices. The population is represented by a \( 3n \)-dimensional point cloud, where each point represents a shape. This cloud can be represented by a mean shape and \( N - 1 \) eigenmode vectors, where the first eigenmode describes the largest variance in the population, the second eigenmode the second largest variance perpendicular to the first, etc. In an SSM, the mean shape \( \bar{\varvec{x}} \in {\mathbb{R}}^{3n} \) and the main shape modes, the principal component (PC) modes \( \varvec{P} \in {\mathbb{R}}^{3n \times (N - 1)} \), are incorporated. This means that a new shape \( \varvec{y} \in {\mathbb{R}}^{3n} \) can be formed by a linear combination of the PCs:

$$ \varvec{y} = \bar{\varvec{x}} + \varvec{Pb}\text{,} $$
(1)

with \( \varvec{b} \in {\mathbb{R}}^{{\left( {N - 1} \right)}} \) the vector containing the SSM parameters.

2.2 Feature Modification

A specific feature of an individual’s shape, such as its height, can be adapted by adding a linear combination of PCs to the individual’s shape vector. The weights for this linear combination, the so-called feature vector, are computed via multiple linear regression of the PC weights on the body features for the population of individuals. By applying a scaled version of the feature vector to the shape vector of an individual, its shape can be adapted to match a specific feature value [8].

Suppose we know \( f \) features \( \varvec{f}_{i} = \left[ {f_{1} \,f_{2} \, \ldots \,f_{f} \;\; 1} \right]^{T} \in {\mathbb{R}}^{f + 1} \) and the PC weights \( \varvec{b}_{i} \in {\mathbb{R}}^{N - 1} \) of each shape \( i \) from the dataset. Then, a mapping matrix \( \varvec{M} \in {\mathbb{R}}^{{\left( {N - 1} \right) \times \left( {f + 1} \right)}} \), describing the relationship between the PC weights matrix \( \varvec{B} = [\varvec{b}_{1} \,\varvec{b}_{2} \, \ldots \,\varvec{b}_{N} ] \in {\mathbb{R}}^{{\left( {N - 1} \right) \times N}} \) and the feature matrix \( \varvec{F} = [\varvec{f}_{1} \,\varvec{f}_{2} \, \ldots \,\varvec{f}_{N} ] \in {\mathbb{R}}^{{\left( {f + 1} \right) \times N}} \) is calculated by [9]:

$$ \varvec{M} = \varvec{BF}^{ + } , $$
(2)

where \( \varvec{F}^{ + } \) is the pseudoinverse of \( \varvec{F} \). With this mapping matrix, new PC weights \( \varvec{b} \) can be generated from given features \( \varvec{f} \) as follows:

$$ \varvec{b} = \varvec{Mf}. $$
(3)

2.3 Identity Removal

Next, the identity of each shape is removed. This means that for each shape, the features are adjusted so that they are equal to the average features. As a result, all shapes look similar.

First, the specific PC weight vector \( \varvec{b}_{i} \) of instance i is extracted from the PC matrix \( \varvec{B} \) as the \( i^{{^{\text{th}} }} \) column vector of \( \varvec{B} \). Second, the delta feature vector \( \Delta \varvec{f} \), which is the vector that holds the values that should be added to the current features to become equal to the average features, is calculated by extracting the specific features \( \varvec{f}_{i} \) (defined as the \( i^{{^{\text{th}} }} \) column of \( \varvec{F} \)) of instance i from the average features \( \bar{\varvec{f}} \) of the population:

$$ \Delta \varvec{f} = \bar{\varvec{f}} - \varvec{f}_{i} . $$
(4)

Next, the delta PC weights vector \( \Delta \varvec{b} \) that should be added to the current PC weights to adjust the body shape, is calculated by multiplying the mapping matrix \( \varvec{M} \) with the calculated delta features \( \Delta \varvec{f} \):

$$ \Delta \varvec{b} = \varvec{M} \cdot\Delta \varvec{f}\text{.} $$
(5)

These delta weights are added to the original PC weights \( \varvec{b}_{i} \) to obtain the PC weights \( \varvec{b}_{i}^{{\prime }} \) of the shape with average features:

$$ \varvec{b}_{i}^{{\prime }} = \varvec{b}_{i} +\Delta \varvec{b}\text{.} $$
(6)

Finally, the new shape without identity, \( \varvec{x}_{i}^{{\prime }} \), is calculated by multiplying the new weights \( \varvec{b}_{i}^{{\prime }} \) with the PC vectors \( \varvec{P} \) of the SSM and adding them to the mean shape \( \bar{\varvec{x}} \):

$$ \varvec{x}_{i}^{{\prime }} = \bar{\varvec{x}} + \varvec{Pb}_{i}^{{\prime }} \text{.} $$
(7)

After removing the identity of each shape, a new SSM is built from the shapes without identity. The result is a posture model, where the shape variances are mainly the posture variances.

2.4 Posture Normalization

To normalize the posture of a shape, it is corresponded with the statistical posture model by elastic surface registration and the PC weights of this shape, using the posture model, are calculated. Because this model mostly contains posture variations, only the posture of the target shape is captured. In Fig. 1, the workflow of posture normalization of an example shape is shown.

Fig. 1.
figure 1

Example of workflow of posture normalization. Note that the arms are more bent in the original shape than in the posture normalized shape

Full size image

First, the input shape, \( \varvec{x}_{j} \), is scaled, so it has the same size as the posture model. Then, the posture PC weights \( \varvec{b}_{j} \) of \( \varvec{x}_{j} \) are calculated by multiplying the inverse PC matrix \( \varvec{P} \) by the distance vector between each vertex of the input surface mesh and the mean surface mesh \( \bar{\varvec{x}} \). These posture PC weights only denote differences in posture compared to the mean posture.

$$ \varvec{b}_{j} = \varvec{P}^{T} (\varvec{x}_{j} - \bar{\varvec{x}}). $$
(8)

The posture \( \varvec{x}_{j}^{{\prime }} \) of the input shape is reconstructed from the calculated posture PC weights \( \varvec{b}_{j} \):

$$ \varvec{x}_{j}^{{\prime }} = \bar{\varvec{x}} + \varvec{Pb}_{j} . $$
(9)

Finally, the normalized shape \( \widehat{\varvec{x}}_{j} \) is calculated by subtracting the posture influence \( \varvec{Pb}_{j} \) on the shape from the original shape \( \varvec{x}_{j} \):

$$ \widehat{\varvec{x}}_{j} = \varvec{x}_{j} - \varvec{Pb}_{\varvec{j}} . $$
(10)

3 Experiments and Results

In this section, the results of the framework are described.

3.1 Statistical Shape Model (SSM)

An SSM was built from 700 subjects (350 men, 350 women) in standing pose from the CAESAR database [3] whose 3D scans were registered using the same template surface mesh, a digitally modeled body consisting of 100k uniformly distributed vertices, shown in Fig. 2. No posture changes were made to these meshes yet.

Fig. 2.
figure 2

Reference mesh, uniformly resampled to 100k vertices.

Full size image

In Fig. 3, the first three PC modes of the SSM built from the original shapes are shown. There is clearly an influence of posture visible in the third mode.

Fig. 3.
figure 3

The first three eigenmodes of the non-normalized SSM, built from the original shapes. A posture variation is clearly noticeable in the third mode, where the position of the arms and shoulders differs.

Full size image

3.2 Posture Model

In Fig. 4, two examples of identity removal are shown. The resulting shapes look more similar than the original shapes.

Fig. 4.
figure 4

Two examples of identity removal. The body features are averaged out, so mainly posture differences remain.

Full size image

The posture model is shown in Fig. 5. The posture model was cut off at 12 shape modes to reduce shape related variations and remove noise from higher modes. This number was empirically determined.

Fig. 5.
figure 5

The first three eigenmodes of the posture model. Posture variation is mainly visible in the region of the arms and torso. For every shape, the average shape is overlaid in light gray to show the difference in posture more clearly.

Full size image

3.3 Posture Normalization Case

To evaluate the performance of the posture normalization algorithm, the posture of an individual from the CAESAR dataset, which was not included in the dataset of the posture model, was three times slightly modified by applying linear blend skinning (LBS) on the surface mesh [4]. Then, the posture of the modified shapes was normalized. In an ideal case, the normalized shapes are equal to each other. The result of the posture deformation and posture normalization is shown in Fig. 6. The average distance between the posture normalized shape was \( (1.08 \pm 3.60) \,{\text{mm}} \).

Fig. 6.
figure 6

Posture normalization case. The same input shape was deformed by applying LBS (upper row). The result of the posture normalization algorithm is shown in the lower row. The average distance between the posture normalized shape is shown in the right figure in mm.

Full size image

Normalizing the posture of a shape took around \( 10\,{\text{s}} \) on a computer with an Intel® Core™ i7-5960X CP @ 3.00 GHz processor and 23.4 GB memory.

3.4 Normalized Statistical Shape Model

Every input shape of the original SSM was corrected for posture. These shapes served as input to build a new, posture normalized, SSM. The first three shape modes of this posture normalized SSM are shown in Fig. 7.

Fig. 7.
figure 7

First three eigenmodes of the posture normalized SSM. The modes describe body shape variances better compared to the shape modes of the non-normalized SSM, shown in Fig. 5. Note that the third mode of the normalized SSM describes mainly gender, while the third mode of the non-normalized SSM describes mainly the position of the arms.

Full size image

3.5 Model Performance – Compactness

Compactness is a widely used measure for quantifying the correspondence quality of an SSM [10, 11]. A compact SSM is a model that can represent body shapes with only a small number of shape modes. The compactness is expressed as the sum of variances of the SSM,

$$ C(m) = \sum\nolimits_{i = 1}^{m} {\uplambda_{\text{i}} ,} $$
(11)

where \( \lambda_{i} \) is the variance in shape mode \( i \), and \( C(m) \) is the compactness using \( m \) modes.

The results are shown in Fig. 8. Note that the normalized SSM is more compact than the non-normalized SSM. To describe more than 90% of the shape variation inside the population, the non-normalized SSM requires seven shape modes, while only two shape modes were sufficient for the normalized SSM. Using only one shape mode, the normalized SSM is \( 16\% \) more compact than the non-normalized SSM. For five shape modes, an improvement of \( 23\% \) was observed, and an improvement of \( 25\% \) using ten shape modes. From 14 shape modes, the difference in compactness converged to \( 27\% \).

Fig. 8.
figure 8

Compactness graph, first 30 shape modes. The average deviation from the mean shape to describe shapes with a specific number of shape modes is shown. The normalized SSM substantially is more compact than the non-normalized SSM and more of its variance is captured with less modes.

Full size image

4 Conclusion and Further Work

In this work, we proposed a technique to perform statistical shape analysis in a posture-invariant way. It allows us to study the shape variations in a database of human body shapes in slightly varying postures. The posture model can be used to normalize any shape that is brought into correspondence with this model in a fast and precise way.

The results have shown that statistical shape analysis of a posture normalized population results in more shape related variations than performing the same analysis on a non-normalized population. The normalized SSM is a more compact representation of the population, compared to the non-normalized shape SSM. Hence, less shape modes are needed to describe a certain percentage of the population.

Our SSM is a valuable tool for product designers for creating more realistic, virtual mannequins, that are a better representation of the population, and using this knowledge to improve the ergonomics of their products. It is also a first step towards posture invariant statistical shape analysis of body shapes in varying poses, e.g. for predicting the body shape in seated pose from a body shape in standing pose by mapping the PC weights from one pose to another.

Further fine-tuning of the posture normalization algorithm is necessary. Therefore, we will test our algorithm on multiple body scans of the same person in slightly different postures to validate and optimize the precision of our posture normalization algorithm.