Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Shape is one of the object characteristics which enables many numerical characterizations suitable for computer supported manipulations. Because of that, different shape concepts are intensively used in object recognition, object identification or object classification tasks. Many approaches to analyse and characterise shapes have been developed. Some of them are very generic, like moment invariants or Fourier descriptors, while others relate to specific object characteristics, e.g. descriptors like convexity, compactness, etc. Another distinction among these approaches is based on which points of shapes are used for analysis. Some approaches use all shape points (area-based ones), other use boundary information only (boundary-based ones), but there are methods which use only specific shape points (convex hull vertices or boundary corners) or hybrid methods (shape compactness is often computed from the relation between shape perimeter and shape area).

In the most research and applications to date, shapes are treated as single objects, even if very often several objects form a group (vehicles on the road, group of people, etc.), and thus, it could be beneficial to consider them as multi-component shapes.

It is not difficult to imagine situations in which it is better to decompose a single shape into its naturally defined components and then, by treating it as a multi-component shape, take some advantage over those methods that treat it as a single shape. There are many more situations where a multi-component shape approach may be appropriate – e.g., when analysing video sequences, multiple instances of objects over time can be grouped together and analysed as a multi-component shape. Several situations where it can be useful to treat objects as multi-component ones are displayed in Fig. 7.1. Here we consider the multi-component shape orientation problem. We overview recently introduced methods (area-based and boundary-based) for the computation of orientation of multi-component shapes and its relation to the most standard shape orientation method (based on the computation of the axis of the least second moment of inertia). As a by-product of the new method for the computation of orientation of compound shapes, an anisotropy measure of such shapes can be derived. This is a first shape measure defined for multi-component shapes and it indicates the consistency of a set of shape component orientations.

Fig. 7.1
figure 1

First row: a group of static objects (flowers), a group of moving objects (birds), a group of different objects (blood cell) make multi-component objects. Second row: the tissue and texture displayed are sometimes better to be decomposed and analysed as multi-component objects. Third row: the appearance of a moving object, in a frame sequence, can be considered and analysed as a multi-component object. Fourth row: different arrangements of a simple object make two multi-component objects with perceptually different orientations [14]

Full size image

All discussions are illustrated with suitably selected experiments. Strict proofs are omitted and for them the readers are referred to the source references.

2 Shape Orientation

Computation of shape orientation is a common problem which appears in a large number of applications in both 2D and 3D, and also in higher-dimensional spaces. Due to the variety of shapes as well as to the diversity of applications, there is no single method for the computation of shape orientation which outperforms the others in all situations. Therefore, many methods have been developed, and different techniques have been used, including those based on complex moments [20], Zernike moments [7], Fourier analysis [1], algebraic arguments [10], etc. The suitability of those methods strongly depends on the particular situation to which they are applied, as they each have their relative strengths and weaknesses. Due to new applications and increasing demands for better computational efficiency, in addition to the previously established methods, there are also many recent examples [3, 5, 11, 12, 16, 24, 27].

Several methods are defined by using a common approach: Consider a suitably chosen function F(S, α) which depends on a given shape S and rotation angle α, and define the orientation of S by the angle which optimizes F(S, α), i.e. by the angle α for which F(S, α) reaches its minimum or maximum. The most standard method for the computation of the shape orientation is such a method. More precisely, this method defines the orientation O st (S) of a given shape S by the, so called, axis of the least second moment of inertia, i.e. by the line which minimises the integral of the squared distances of the shape points to the line (see Fig. 7.2). Simple algebraic manipulation shows that such a line passes through the shape centroid. Note that the centroid of a given shape S is defined as

$$\left (x_{S},y_{S}\right ) = \left (\frac{\displaystyle\int \displaystyle\int _{S}xdxdy} {\displaystyle\int \displaystyle\int _{S}dxdy} \frac{\displaystyle\int \displaystyle\int _{S}ydxdy} {\displaystyle\int \displaystyle\int _{S}dxdy} \right ).$$
(7.1)

So, in order to compute the orientation of a given shape S it is sufficient to find the minimum of the function

$$F_{st}(S,\alpha ) =\displaystyle\int \displaystyle\int _{S}r{(x,y,\alpha )}^{2}dxdy$$
(7.2)

where r(x, y, α)2 is the perpendicular distance of the point (x, y) to the line which passes thought the centroid \(\left (x_{S},y_{S}\right )\) of S and has a slope α. If we assume that the centroid of S coincides with the origin, i.e., \(\left (x_{S},y_{S}\right ) = (0,0),\) r(x, y, α)2 becomes (x ⋅sinα − y ⋅cosα)2, and the optimizing function F st (S, α) in (7.2) can be expressed as

$$\displaystyle\begin{array}{rcl} F_{st}(S,\alpha )& =& \displaystyle\int \displaystyle\int _{S}r{(x,y,\alpha )}^{2}dxdy \\ & =& \mu _{2,0}(S) {\cdot \sin }^{2}\alpha +\mu _{ 0,2}(S) {\cdot \cos }^{2}\alpha -\mu _{ 1,1}(S) \cdot \sin (2\alpha ),\end{array}$$
(7.3)

where μ p, q (S) are the well-known centralised geometric moments [21] defined, for all p, q ∈ { 0, 1, 2, …}, as

$$\mu _{p,q}(S) =\displaystyle\int \displaystyle\int _{S}{(x - x_{S})}^{p}{(y - y_{ S})}^{q}dxdy.$$
(7.4)
Fig. 7.2
figure 2

The standard method defines the orientation of a given shape by the line which minimizes the integral of squared distances of the shape points to the line

Full size image

Now, we come to the following definition of the orientation of a given shape.

Definition 7.1.

The orientation of a given shape S is determined by the angle α where the function F st (S, α) reaches its minimum.

This standard method defines the shape orientation in a natural way – by the line which minimizes the integral of the squared distances of shape points to this line. Such a definition matches our perception of what the shape orientation should be. Also, there is a simple formula for the computation of such orientation. It is easy to check [21] that the angle which minimizes F st (S, α) satisfies the equation

$$\tan (2 \cdot O(S)) = \frac{2 \cdot \mu _{1,1}(S)} {\mu _{2,0}(S) -\mu _{0,2}(S)}.$$
(7.5)

These are desirable properties, but there are some drawbacks too. The main problem is that there are many situations where the method fails [23, 28] or does not perform satisfactorily. The situations were the method fails are easy to characterise. Indeed, considering the first derivative of F st (S, α) (see (7.3))

$$\frac{dF_{st}(S,\alpha )} {d\alpha } = (\mu _{2,0}(S) -\mu _{0,2}(S)) \cdot \sin (2\alpha ) - 2\mu _{1,1}(S) \cdot \cos (2\alpha ),$$
(7.6)

and looking for the conditions when dF st (S, α) ∕ dα vanishes, it is easy to see that for all shapes S satisfying

$$\mu _{2,0}(S) -\mu _{0,2}(S) = 0\qquad \qquad \mbox{ and}\qquad \qquad \mu _{1,1}(S) = 0$$
(7.7)

the function F st (S, α) is constant and consequently does not tell which angle should be selected as the shape’s orientation. N-fold rotationally symmetric shapes are shapes which satisfy (7.7) but there are many other (irregular) shapes which satisfy (7.7) and consequently could not be oriented by the standard method given by Definition 7.1.

In order to overcome such problems, [23] suggested a modification of the optimizing function F st (S, α) by increasing the exponent in (7.2). The method from [23] defines the orientation of a given shape S whose centroid coincides with the origin, by the angle which minimizes

$$F_{N}(S,\alpha ) =\displaystyle\int \displaystyle\int _{S}r{(x,y,\alpha )}^{2N}dxdy$$
(7.8)

for a certain exponent 2N. In such a way, the class of shapes whose orientation can be computed is expanded. On the other hand, there is not a closed formula (analogous to (7.5)) for the computation of the shape orientation by using F N (S, α), for an arbitrary N.

Notice that difficulties in the computation of the shape orientation can be caused by the nature of certain shapes. While for many shapes their orientations are intuitively clear and can be computed relatively easily, the orientation of some other shapes may be ambiguous or ill defined. Problems related to the estimation of the degree to which a shape has a distinct orientation are considered in [29].

3 Orientation of Multi-component Shapes

As discussed before, there are many methods for the computation of the orientation of single-component shapes. On the other hand, as mentioned earlier, in many situations, several single objects usually appear as a group (e.g. the shoal of fish in Fig. 7.4, flock of birds in Fig. 7.1, vehicles on the road, etc.). Also, in many situations, it is suitable to consider a single object as a multi-component one, consisting of suitably defined components (as cells in embryonic tissue displayed in Fig. 7.1, or material micro structure elements, etc.). In addition, the appearances of the same object at different times can be also considered as components of a multi-component shape. Some examples where treating objects as multi-component shapes becomes very natural are in Fig. 7.1 and also in the forthcoming experiments.

Fig. 7.3
figure 3

The orientation of multi-component shapes is defined by the direction α which minimizes the integral of squared projections of line segments whose end points belong to a certain component

Full size image
Fig. 7.4
figure 4

Real images are in the first row. After thresholding, the image components are treated as components of a multi-component shape and then oriented by the method given by Definition 7.2 – the orientations computed are presented by short dark arrows. Long grey arrows represent the orientations computed by the standard method where all components are taken together to build a single shape

Full size image

In this section we consider the method for the computation of the orientation of multi-component shapes introduced by Žunić and Rosin [26]. Before that, note that most of the existing methods for the computation of the orientation of single component shapes do not have a (at least straightforward) extension which can be used to compute the orientation of compound shapes. The main reason for this is that most of the existing methods have a 180 ∘  ambiguity about the computed orientation. That is because they define the shape orientation by a line, not by a vector. Thus, the orientations of \(\varphi\) degrees and the orientation of \(\varphi +18{0}^{\circ }\) are considered to be the same. A consequence of such an ambiguity is that natural ideas how to compute the orientation of a multi-component shape from the orientations assigned to its components, do not work. For example, if S 1, S 2, …, S n are components of a multi-component shape S, then most of the existing methods would compute their orientations as \(\varphi _{1} + a_{1} \cdot 18{0}^{\circ },\) \(\varphi _{2} + a_{2} \cdot 18{0}^{\circ },\) …, \(\varphi _{n} + a_{n} \cdot 18{0}^{\circ },\) where the numbers a 1, a 2, …, a n are arbitrarily chosen from {0, 1}. Thus if, in the simplest variant, the orientation of multi-component shape \(S = S_{1} \cup S_{2} \cup \ldots \cup S_{n}\) is computed as the average value of the orientations assigned to its components, then the orientation of S would be computed as

$$\frac{(\varphi _{1} + a_{1} \cdot 18{0}^{\circ }) +\ldots +(\varphi _{n} + a_{n} \cdot 18{0}^{\circ })} {n} = \frac{\varphi _{1} +\ldots +\varphi _{n}} {n} \; +\; \frac{(a_{1} +\ldots +a_{n}) \cdot 18{0}^{\circ }} {n}$$

and obviously, for different choices of a 1, a 2, …, a n , the computed orientations are inconsistent (i.e. they could differ for an arbitrary multiple of the fraction 180 ∘  ∕ n). This is obviously unacceptable.

Now we consider the method for the computation of multi-component shapes described in [26]. The authors define the orientation of a multi-component shape by considering the integrals of the squared length of projections of all edges whose end points belong to a certain component. Before a formal definition, let us introduce the necessary denotations (see Fig. 7.3 for an illustration):

  1. –

    Let \(\overrightarrow{a} = (\cos \alpha ,\sin \alpha )\) be the unit vector in the direction α; 

  2. –

    Let \(\vert \mathbf{pr}_{\overrightarrow{a}}[AB]\vert \) denote the length of the projection of the straight line segment [AB] onto a line having the slope α.

Definition 7.2.

Let S be a multi-component shape which consists of m disjoint shapes S 1, S 2, …, S m . Then the orientation of S is defined by the angle that maximises the function G comp (S, α) defined by

$$G_{\mathit{c}omp}(S,\alpha ) =\displaystyle\sum _{ i=1}^{m}\;\;\displaystyle\int \displaystyle\int \limits _{{ A=(x,y)\in S_{i} \atop B=(u,v)\in S_{i}} }\displaystyle\int \displaystyle\int \vert \mathbf{pr}_{\overrightarrow{a}}[AB]{\vert }^{2}dx\ dy\ du\ dv.$$
(7.9)

The previous definition is naturally motivated, but also enables a closed formula for the computation of the orientation of multi-component shapes. This is the statement of the following theorem.

Theorem 7.1.

The angle α where the function G comp (S, α) reaches its maximum satisfies the following equation

$$\frac{\sin (2\alpha )} {\cos (2\alpha )} = \frac{2 \cdot \displaystyle\sum \limits _{i=1}^{m}\mu _{1,1}(S_{i}) \cdot \mu _{0,0}(S_{i})} {\displaystyle\sum \limits _{i=1}^{m}(\mu _{2,0}(S_{i}) -\mu _{0,2}(S_{i})) \cdot \mu _{0,0}(S_{i})}.$$
(7.10)

To prove the theorem it is sufficient to enter the following two trivial equalities

$$\vert \mathbf{pr}_{\overrightarrow{a}}[AB]{\vert }^{2} = {((x - u) \cdot \cos \alpha +(y - v)\cdot \sin \alpha )}^{2},\mbox{ for }A = (x,y),B = (u,v)$$
(7.11)

and

$$\displaystyle\int \displaystyle\int \limits _{S\times S}\displaystyle\int \displaystyle\int {x}^{p}{y}^{q}{u}^{r}{v}^{t}\ dx\ dy\ du\ dv =\mu _{ p,q}(S) \cdot \mu _{r,t}(S)$$
(7.12)

into the optimizing function G comp (S, α). After that, simple calculus applied to the equation \(\frac{dG_{\mathit{c}omp}(S,\alpha )} {d\alpha } = 0\) establishes the proof. For more details we refer to [26].

The orientation of multi-component shapes computed by optimizing the function G comp (S, α) is theoretically well founded and because of that it can be well understood. Some of properties are a direct consequence of the definition and can be proved by using basic calculus. We list some of them.

Property 7.1.

If the method given by Definition 7.2 is applied to a single component shape, then the computed orientation is the same as the orientation computed by the standard definition, i.e. by the optimizing the function F st (S, α) (see (7.2) and (7.5)). Note that the optimizing function G(S, α) = G comp (S, α), specified for single component shapes, and the optimizing function F st (S, α) are different, but they are connected with the following, easily provable, equality

$$G(S,\alpha ) + 2 \cdot \mu _{0,0}(S) \cdot F_{st}(S,\alpha ) = 2 \cdot \mu _{0,0}(S) \cdot (\mu _{2,0}(S) +\mu _{0,2}(S)).$$
(7.13)

Since the right-hand side of (7.13) does not depend on α we deduce that the maximum of G(S, α) and the minimum of F st (S, α) are reached at the same point. In other words, the direction α which defines the orientation of S by applying the standard method is the same as the direction which defines the orientation of S if the Definition 7.2 is applied to single component shapes.

Property 7.2.

As it is expected, there are situations where the method given by Definition 7.2 fails. Due to the definition of the optimizing function G comp (S, α), a simple characterization of such situations is possible. Indeed, by using (7.11) we deduce:

$$\displaystyle\begin{array}{rcl} G_{\mathit{comp}}(S,\alpha )& =& {\cos }^{2}\alpha \cdot \displaystyle\sum _{ i=1}^{m}2\mu _{ 0,0}(S_{i})\mu _{2,0}(S_{i}) +{\sin }^{2}\alpha \cdot \displaystyle\sum _{ i=1}^{m}2\mu _{ 0,0}(S_{i})\mu _{0,2}(S_{i}) \\ & & +\sin (2\alpha ) \cdot \displaystyle\sum _{i=1}^{m}2\mu _{ 0,0}(S_{i})\mu _{1,1}(S_{i}). \end{array}$$
(7.14)

The last equality says immediately that the first derivative \(\frac{dG_{\mathit{c}omp}(S,\alpha )} {d\alpha }\) is identically equal to zero (i.e. G comp (S, α) is constant) if and only if the following two conditions are satisfied

$$\displaystyle\sum \limits _{i=1}^{m}\mu _{ 0,0}(S_{i}) \cdot \mu _{1,1}(S_{i}) = 0\;\;\;\;\mbox{ and }\;\;\;\;\displaystyle\sum \limits _{i=1}^{m}\mu _{ 0,0}(S_{i}) \cdot \left (\mu _{2,0}(S_{i}) -\mu _{0,2}(S_{i})\right ) = 0.$$
(7.15)

So, under the conditions in (7.15) the optimizing function is constant and no direction can be selected as the shape orientation.

The Eq. (7.14) also says that the components S i of a multi-component shape \(S = S_{1} \cup \ldots \cup S_{m}\) which satisfy

$$\mu _{1,1}(S_{i}) = 0\;\;\;\;\mbox{ and }\;\;\;\;\mu _{2,0}(S_{i}) -\mu _{0,2}(S_{i}) = 0,$$

(i.e. the shapes are not orientable by the standard method) do not contribute to the G comp (S, α) and because of that, such components S i , can be omitted when computing the orientation of S.

Notice that in case of G comp (S, α) = constant we can increase the exponent in (7.9) and define the orientation of S by the direction which maximizes the following function

$$G_{comp,N}(S,\alpha ) =\displaystyle\sum _{ i=1}^{m}\;\;\displaystyle\int \displaystyle\int \limits _{{ A=(x,y)\in S_{i} \atop B=(u,v)\in S_{i}} }\displaystyle\int \displaystyle\int \vert \mathbf{pr}_{\overrightarrow{a}}[AB]{\vert }^{2N}dx\ dy\ du\ dv.$$

In this way the class of multi-component shapes whose orientation is well defined would be extended. A drawback is that there is no closed formula (similar to (7.10)) which enables easy computation of such a defined multi-component shape orientation.

The next property seems to be a reasonable requirement for all methods for the computation of the orientation of multi-component shapes.

Property 7.3.

If all components S i of \(S = S_{1} \cup \ldots \cup S_{m}\) have an identical orientation α, then the orientation of S is also α. 

To prove the above statement it is sufficient to notice that if all components S i have the same orientation (see Property 7.1), then there would exist an angle α 0 such that all summands

$$\displaystyle\int \displaystyle\int \limits _{{ A=(x,y)\in S_{i} \atop B=(u,v)\in S_{i}} }\displaystyle\int \displaystyle\int \vert \mathbf{pr}_{\overrightarrow{a}}[AB]{\vert }^{2}dx\ dy\ du\ dv.$$
(7.16)

in (7.9) reach their maximum for the angle α = α 0. An easy consequence is that α = α 0 optimizes \(G_{\mathit{c}omp}(S,\alpha ) =\sum _{ i=1}^{m}\;\int \int \limits _{{ A=(x,y)\in S_{i} \atop B=(u,v)\in S_{i}} }\int \int \vert \mathbf{pr}_{\overrightarrow{a}}[AB]{\vert }^{2}dx\ dy\ du\ dv,\) as well. This means that the orientation of S coincides with the orientation of its components S i . 

Property 7.4.

The method established by Definition 7.2 is very flexible, in that the influence of the shape component’s size (to the computed orientation) can vary. In the initial form in Definition 7.2 the moments \(\mu _{1,1}(S_{i}),\) \(\mu _{2,0}(S_{i})\) and \(\mu _{0,2}(S_{i})\) are multiplied with the size/area of S i , i.e. by \(\mu _{0,0}(S_{i})\). But the method allows \(\mu _{0,0}(S_{i})\) to be replaced with \(\mu _{0,0}{(S_{i})}^{T}\) for some suitable choice of T. In this way the influence of the shape components to the computed orientation can be controlled. The choice \(T = -2\) is of a particular importance. In this case the size/area of the shape components does not have any influence to the computed orientation. This is very suitable since objects, which are of the same size in reality, often appear in images at varying scales since their size depends on their relative position with respect to the camera used to capture the image.

3.1 Experiments

This subsection includes several experiments which illustrate how the method for the computation of the orientation of multi-component shapes works in practice. Since this is the first method for the computation of the orientation of such shapes, there are no suitable methods for comparison. In the experiments presented, together with the orientations computed by the new method, the orientations computed by the standard method, which treats all the multi-component objects as a single object, are also displayed. However, orientations computed by the standard method are displayed just for illustrative purposes, not for qualitative comparison against the new method.

In the first example in Fig. 7.4 three images are displayed. In all three cases the objects that appear (humans, fish and blood cell components) are treated as components of a multi-component shape (a group of people, a shoal, a blood sample) and are then oriented. In the case of the group of people and the fish shoal the computed orientations (by the new method) are in accordance with our perception. As expected the computation by the standard method (i.e. treating the multi-component shapes as single ones) does not lead to orientations which match our perception. The same could be said for the third example (blood cell) even though the our perception of what the orientation should be is not as strong as in the first two examples.

The next figure illustrates a very nice and useful property of the new method. It illustrates that for multi-component shapes whose components are relatively consistently oriented, the computed orientation of a subset of such shapes coincides with the orientation of the whole shape. Somehow it could be said that the computed orientations do not depend much on the frame used to capture a portion of the multi-component shape considered. The humans and fish shoal images (in Fig. 7.5) are split onto two halves. The halves are treated as multi-component shapes and oriented by the new method. The computed orientations are shown by the short dark arrows. As it can be seen such computed orientations are consistent – i.e. the orientations of the halves coincide with the orientation of the whole multi-component shape. As expected, the orientations computed by the standard method (long grey arrows) do not coincide. The third example in this subsection illustrates a possible application of the method to the orientation of textures. Wood textures, displayed in the first row in Fig. 7.6, are not multi-component objects with clearly defined components. Nevertheless, after suitable thresholding the components become apparent, and the orientation of such obtained multi-component shapes can be computed. The results are in the third row and it could be said (in the absence of ground truth) that the orientations obtained are in accordance with our perception.

Fig. 7.5
figure 5

The orientation computed by the method from Definition 7.2 for halves and the whole shape illustrate the “Frame independence” property of the new method for the orientation of multi-component shapes

Full size image
Fig. 7.6
figure 6

Texture images, in the first row, are thresholded and their orientation is then computed as the orientation of multi-component shapes which correspond to the black-and-white images in the second row. The images in the third row are re-oriented in accordance with the computed orientations

Full size image

The last example in this subsection is somewhat different from the previous ones. In this case, a gait sequence is taken from NLPR Gait Database [25] and each appearance of a human silhouette in the sequence of the frames is considered as a component of the multi-component shape analysed. So, in this case the shape components are distributed temporally across the sequence (not spatially over the image, as in the previous examples). After segmentation many errors, which typically appear, have been removed using standard morphological techniques. However, several large errors remain and the task was to detect them. Due to the nature of the shapes it is expected that all components (i.e. silhouettes) are fairly consistently oriented if they are extracted properly. Thus, we make the hypothesis that the silhouettes with orientations inconsistent with the majority of silhouettes suffer from segmentation errors. The difference in multi-component shape orientation caused by removing the least consistent component has been used as a criterion to find possible outliers. In the example given, due to the errors in the processing chain that produced the sequence of binary images, the person’s leading leg has been displaced and this silhouette/frame has been detected as an outlier, as shown in Fig. 7.7.

Fig. 7.7
figure 7

The extracted silhouettes from a gait sequence are displayed (the first row) and underneath is an intensity coding of each silhouette to show its degree of being an outlier (dark means high likelihood). A magnified view of the most outlying silhouette and its neighbours is in the third row

Full size image

4 Boundary-Based Orientation

So far we have considered area-based methods only, i.e. the methods which use all the shape points for the computation of (in this particular case) shape orientation. But boundary-based methods for the computation of shape orientation are also considered in the literature. Apart from the fact that some area-based methods for the computation of the shape orientation have a straightforward extension to their boundary-based analogues, there are some boundary-based methods which cannot be derived in such a way – some examples are in [12, 27].

The methods considered in the previous section have an easy extension to boundary-based methods. For example, an analogue to the standard method for the computation of shape orientation is the method which orients a given shape by the line which minimizes the integral of the squared distance of the boundary points to this line. Since only the boundary points are used for the computation, this method is considered as a boundary-based one. The line which minimizes the optimizing integral can be obtained by following the same formalism as in the case of the standard method, but the appearing area integrals should be replaced by line integrals.

So, first we have to place S such that its boundary-based centroid coincides with the origin. The boundary-based centroid \((x_{\partial S},y_{\partial S})\) is defined as the average of the boundary points. This means that

$$\left (x_{\partial S},y_{\partial S}\right ) = \left (\frac{\displaystyle\int _{\partial S}x(s)ds} {\displaystyle\int _{\partial S}ds} \frac{\displaystyle\int _{\partial S}y(s)ds} {\displaystyle\int _{\partial S}ds} \right ).$$
(7.17)

where the boundary ∂S of S is given in an arc-length parametrization: x = x(s), y = y(s), \(s \in [0,perimeter\text{\_}of\text{\_}S].\) Note that such a choice of the boundary representation is suitable because it preserves rotational invariance – i.e. if the shape is rotated for a certain angle, then the computed orientation is changed by the same angle. In the rest of this chapter an arc-length parametrization of the appearing boundaries/curves will be always assumed, even not mentioned.

Next, in order to compute the boundary-based orientation of a given shape S, we have to find the minimum of the function

$$L_{bo}(\partial S,\alpha ) =\displaystyle\int _{\partial S}r{(x,y,\alpha )}^{2}ds$$
(7.18)

where r(x, y, α) is, as in (7.2), the orthogonal distance of the point (x, y) to the line passing through the origin and having the slope α. 

The optimizing function L bo (∂S, α) can be expressed as

$$L_{bo}(\partial S,\alpha )\; =\;\nu _{2,0}(\partial S) {\cdot \sin }^{2}\alpha +\nu _{ 0,2}(\partial S) {\cdot \cos }^{2}\alpha -\nu _{ 1,1}(\partial S) \cdot \sin (2\alpha ),$$
(7.19)

where ν p, q (∂S) are the normalised line moments [2] defined as

$$\nu _{p,q}(\partial S) =\displaystyle\int _{\partial S}{(x - x_{\partial S})}^{p}{(y - y_{ \partial S})}^{q}ds,$$
(7.20)

for all p, q ∈ { 0, 1, 2, …}.

Finally, the maximum of the optimizing function L bo (∂S, α) is reached for α = O bo (∂S) which satisfies the following equation:

$$\tan (2 \cdot O_{bo}(\partial S)) = \frac{2 \cdot \nu _{1,1}(\partial S)} {\nu _{2,0}(\partial S) -\nu _{0,2}(\partial S)}.$$
(7.21)

The equality (7.21) is an obvious analogue for the equality (7.5) related to the standard area-based method. This is as expected because the same formalism is used in both area-based and boundary-based cases.

The idea used in Sect. 7.3 to define the orientation of multi-component shapes also has a boundary-based analogue. The problem is studied in detail in [15]. Following the idea from [26], the authors consider the projections of the edges whose end points belong to the boundary of a certain component of a given multi-component shape and define the shape orientation by the line which maximises the integral of the squared values of the projections of such edges to the line. The formal definition follows.

Definition 7.3.

Let \(S = S_{1} \cup \ldots \cup S_{m}\) be a multi-component shape and let the boundary of S be the union of the boundaries ∂S i of the components of S: \(\partial S = \partial S_{1} \cup \ldots \cup \partial S_{m}.\) The orientation of S (Fig. 7.8) is defined by the angle that maximises the function L comp (∂S, α) defined as follows

$$L_{\mathit{c}omp}(\partial S,\alpha ) =\displaystyle\sum _{ i=1}^{m}\;\;\displaystyle\int \limits _{{ s\in [0,per(S_{i})] \atop l\in [0,per(S_{i})]} }\displaystyle\int \vert \mathbf{pr}_{\overrightarrow{a}}[A_{i}(s)B_{i}(l)]{\vert }^{2}ds\ dl.$$
(7.22)

Definition 7.3 enables easy computation of the angle which defines the orientation of ∂S, as given by the following theorem.

Theorem 7.2.

Let S be a multi-component shape whose boundary is \(\partial S\,=\,\partial S_{1} \cup \ldots \cup \partial S_{m}\). The angle α where the function L comp (∂S, α) reaches its maximum satisfies the following equation

$$\tan (2\alpha ) = \frac{2 \cdot \displaystyle\sum \limits _{i=1}^{m}\nu _{1,1}(\partial S_{i}) \cdot \nu _{0,0}(\partial S_{i})} {\displaystyle\sum \limits _{i=1}^{m}(\nu _{2,0}(\partial S_{i}) -\nu _{0,2}(\partial S_{i})) \cdot \mu _{0,0}(\partial S_{i})}.$$
(7.23)

Being derived in an analogous way as the area-based method for the computation of the multi-component shapes, the method given by Definition 7.3 also satisfies the Properties 7.1–7.4 listed in Sect. 7.3, before the experimental section. It is worth mentioning that the method also enables the control of the influence of the perimeter of the shape components to the computed orientation. If, for some applications, such an influence should be ignored then the moments \(\nu _{1,1}(\partial S_{i}),\) \(\nu _{2,0}(\partial S_{i}),\) and \(\nu _{0,2}(\partial S_{i})\) which appear in (7.23) should be multiplied by \({(\nu _{0,0}(\partial S_{i}))}^{-2}\).

Fig. 7.8
figure 8

On the left: The standard method defines the orientation of the shape by the line which minimizes the integral of squared distances of the shape boundary points to the line. On the right: the boundary-based method for computation of the orientation of multi-component shapes considers the projections of edges whose end points belong to the boundary of a certain component

Full size image

To close this section, let us mention that (in general) boundary-based approaches to define the orientation of shapes allow some extra generalizations. One such generalisation was considered in [12]. Therein the shape orientation is computed based on the projections of the tangent vectors at the shape boundary points, weighted by the suitably chosen function of the boundary curvature at the corresponding points. The advantage of such an approach is not only that the boundary curvature, as an important shape feature, is involved in the computation of the shape orientation. This also provides an easy approach to overcome situations were the orientations are not computable. It is sufficient to modify the curvature based weighting function, and shapes which were not “orientable” by an initial choice of the weighting function can become orientable with another choice. The computation of the shape orientation remains possible by a closed form formula whether the weighting function is changed or not. For more details see [12].

4.1 Experiments

In this section we have two examples to illustrate how the boundary-based method (established by Definition 7.3) works.

First we consider the embryonic tissue displayed in Fig. 7.9. Cell boundaries are extracted and then the boundary-based method for the computation of the orientation of multi-component shapes is applied. The orientations are computed for the whole image and also separately for the upper and lower parts. The computed orientations are consistent (presented by short dark arrows) which actually implies that the tissue displayed has an inherent consistent orientation. The orientations computed by the boundary-based analogue (by optimizing L bo (∂S, α) from (7.18)), are different (long grey arrows), as it was expected.

Fig. 7.9
figure 9

Boundaries of cells of an embryonic tissue (on the left) are extracted, and then split onto an “upper” and “lower” part (on the right). Orientations computed by optimizing L comp (∂S, α) all coincide (short dark arrows). The orientations by the analogue of the standard method are shown by long grey arrows

Full size image

In the second example the boundary-based method for multi-component shapes (Definition 7.3) has been used to compute the orientations of signatures. Better results were obtained than when the orientations were computed by the standard boundary-based method. Figure 7.10 displays signatures of subject s048 from Munich and Perona [13]. In the first row the signatures are oriented by applying the standard boundary-based method and the problems are obvious. Signatures are not oriented consistently, which is a problem because the similarity measure used in [13] was not rotationally invariant. In the second row are the same signatures but oriented by the boundary-based multi-component method. Prior to the computation of the orientation, the signatures were segmented at the vertices with small subtended angles. The obtained orientations are very consistent, as required.

Fig. 7.10
figure 10

Figure: Signatures of subject s048 from Munich and Perona [13]. Signatures in the top row are re-oriented according to the standard method while the bottom row displays the same signatures re-oriented according to the multi-component method

Full size image

5 Anisotropy of Multi-component Shapes

In all the methods presented here, shape orientation is computed by optimizing a suitably chosen function which depends on the orientation angle and the shape considered. Depending on the method selected, either the angle which maximizes the optimizing function or the angle which defines the minimum of the optimizing function is selected as the shape orientation. To have a reliable method for orienting the shape it is important to have distinct minima and maxima of the optimizing function. This is because in image processing and computer vision tasks we deal with discrete data and very often in the presence of noise. Thus, if the optima of the optimizing function are not distinct significant values then the computed orientations could arise due to noise or digitization errors, rather than from inherent properties of the shapes. Note that for a fixed shape the difference of maxima and minima of the optimizing function could dependent strongly on the method applied. That is why most of the methods only suit certain applications well. The question of whether a shape possess an inherent orientation or not is considered in [29]. The authors have introduced a shape orientability measure as a quantity which should indicate to which degree a shape has a distinct orientation.

The ratio \(\mathcal{E}_{st}(S)\) of the maxima and minima of the optimizing function F st (S, α), in the case of the standard method, is well studied and widely used in shape based image analysis. The quantity

$$\mathcal{E}_{st}(S) = \frac{\mu _{2,0}(S) +\mu _{0,2}(S) + \sqrt{4 \cdot {(\mu _{1,1 } (S))}^{2 } + {(\mu _{2,0 } (S) -\mu _{0,2 } (S))}^{2}}} {\mu _{2,0}(S) +\mu _{0,2}(S) -\sqrt{4 \cdot {(\mu _{1,1 } (S))}^{2 } + {(\mu _{2,0 } (S) -\mu _{0,2 } (S))}^{2}}}.$$
(7.24)

is well-known as the shape elongation measure. \(\mathcal{E}_{st}(S)\) ranges over [1, ∞) and takes the value 1 if S is a circle. A problem is that there are many other shapes whose elongation equals 1. In addition, \(\mathcal{E}_{st}(S)\) is invariant with respect to translation, rotation and scaling transformations and is easily and accurately computable from the object images [8, 9].

The elongation measure \(\mathcal{E}_{st}(S)\) has its boundary-based analogue – the area moments in (7.24) have to be replaced with the corresponding line integrals along the shape boundaries. Another elongation measure is suggested by Stojmenović and Žunić [22].

Another related property is shape anisotropy. It has a natural meaning for single component shapes. For example, for a shape centred at the origin in both 2D and 3D, it could be inversely related to the degree to which the shape points are equally distributed in all directions [18, 19]. It has been used as a useful feature in shape (object) classification tasks but also as a property which highly correlates with some mechanical characteristics of certain real objects and materials [4, 6]. It is also of interest when analyzing tracks of different species of animals [17].

The anisotropy measure of multi-component shapes has not been considered before, but it seems that it should be given different meaning than in the case of single component shapes. Our understanding is that an anisotropy measure for the multi-component shapes should indicate how consistently the shape’s components are oriented. It has turned out that a quantity defined as the ratio between the maxima and minima of the function L comp (∂S, α) from Definition 7.3 provides such a measure. So, we give the following definition.

Definition 7.4.

Let S 1, …, S m be components of a compound shape S. Then the anisotropy \(\mathcal{A}(S)\) of S is defined as

$$\mathcal{A}(S)\; =\; \frac{\max _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S,\alpha )} {\min _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S,\alpha )}\; =\; \frac{\max _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S_{1} \cup \ldots \cup S_{m},\alpha )} {\min _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S_{1} \cup \ldots \cup S_{m},\alpha )}.$$
(7.25)

The anisotropy measure \(\mathcal{A}(S)\) for multi-component shapes ranges over [1, ∞) and is invariant with respect to translations, rotations and scaling transformations. It also enables an explicit formula for its computation. By basic calculus it can be shown that the maxima and minima of L comp (∂S, α) are:

$$\max _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S,\alpha ) = C + \sqrt{{A}^{2 } + {B}^{2}},\qquad \min _{\alpha \in [0,2\pi )}L_{\mathit{c}omp}(\partial S,\alpha ) = C -\sqrt{{A}^{2 } + {B}^{2}}$$

where the quantities A, B, and C are

  • \(A =\sum \limits _{ i=1}^{m}(\nu _{2,0}(\partial S_{i}) -\mu _{0,2}(\partial S_{i})) \cdot \nu _{0,0}(\partial S_{i}),\)

  • \(B =\sum \limits _{ i=1}^{m}2 \cdot \nu _{1,1}(\partial S_{i}) \cdot \nu _{0,0}(\partial S_{i}),\)

  • \(C =\sum \limits _{ i=1}^{m}(\nu _{2,0}(\partial S_{i}) +\nu _{0,2}(\partial S_{i})) \cdot \nu _{0,0}(\partial S_{i}).\)

We illustrate how the anisotropy measure \(\mathcal{A}(S)\) acts by two examples. Notice that the anisotropy measure, as defined here, also depends on the elongation of the shape components, not only on their orientations. This also seems acceptable, e.g. a stereotype for a multi-component shape with a high anisotropy \(\mathcal{A}(S)\) is a shape whose components have high elongations and the same orientation.

The first example is in Fig. 7.11. The shapes in both rows are treated as a multiple component object (e.g. a herd of cattle and a group of cars). The anisotropy was first computed for just the cattle, giving a value of 3. 49. The anisotropy of the cattle’s shadows alone increases to 7. 57 since the shadows are more consistently orientated, and are also slightly more elongated. Merging the cattle and their shadows produces even more elongated regions, which lead to an increase of the herd’s anisotropy to 12.08. The anisotropy of the cars (in the second row) is 1. 38, which is very small. This is to be expected since the orientations of the individual cars vary strongly.

Fig. 7.11
figure 11

Object boundaries are extracted from the original images and considered as components of the boundary of a multi-component shape. The highest anisotropy of 12. 08 is measured if the cattle and their shadows are merged and considered as individual components of multi-component shapes. A low anisotropy measure of 1. 38 is computed for the compound shape in the second row

Full size image

The second example which indicates how the anisotropy measure \(\mathcal{A}(S)\) acts is in Fig. 7.12, in which anisotropy is used to select appropriate elongated regions to enable skew correction of the document. The first image (on the left) in the first row is the original image. Its components are letters whose orientations vary strongly, but also many of the letters have a low elongation. This results in a very low anisotropy of this image, as it can be seen from the graph given in the second row.

Fig. 7.12
figure 12

The image on the left is blurred and thresholded, and the resulting component anisotropy is plotted. Three of the thresholded images are shown demonstrating that maximum anisotropy is achieved when many of the words are merged into lines

Full size image

After blurring is applied to the image the characters start to merge into words, which are both more consistently oriented and more elongated. This leads to an increase of the anisotropy (see the second image in the first row). If enough blurring is applied to merge characters/words into continuous lines the anisotropy increases dramatically (see the third image in the first row).

More blurring is counter productive to the task of skew correction, as sections of adjacent lines merge, and their anisotropy quickly drops (the last image in the first row).

6 Conclusion

Multi-component shapes have not been considered much in literature. This is somewhat surprising since there are many situations in which objects act as a part of a very compact group, or where single objects need to be decomposed onto components for analysis. There are also some less obvious situations were the multi-component approach can be useful.

We have focused on two problems related to multi-component shapes: computing orientation and anisotropy. Both problems have only recently been considered [15, 26] and some solution were offered. These problems do not have analogues in the existing single-component shape based methods. Thus, new ideas have to be developed.

The obtained results are promising, and have been justified with a number of experiments. The extension to the other shape based techniques will be investigated in the future. Due to the variety of ways that a multi-component shape can be defined, there are plenty of different demands which have to be satisfied by the methods developed. Moreover, there are some specific demands which do not exist when dealing with single-component shapes. To mention just two of them, which have been discussed in this chapter: the frame independence property and the tunable influence of the component size to the method performance.