Keywords

1 Introduction

The article is devoted to mathematical and functional/physical interpretation of image analysis and processing operations used as sets of operations (ring elements) of descriptive image algebras (DIAs) [3, 4, 6,7,8].

This article continues the study of interpretability of DIA operations begun in [8].

DIAs are studied in the framework of developing a mathematical apparatus for analyzing and evaluating information in the form of images. For a structured description of possible algorithms for solving these problems, a formal tool is needed to describe and validate the chosen solution path. For the formalization, an algebraic apparatus was chosen [3] that should ensure the uniformity of procedures for describing image objects and transformations over these image objects.

Despite a number of significant works in the field of “algebraization” of image processing, analysis and recognition, it can be argued that there is currently no generally accepted unified approach to solving problems in this subject area.

In the late 1980s and 1990s, Gurevich [3,4,5,6,7,8,9] specialized a general algebraic approach to solving recognition, classification and prediction problems [10] (Zhuravlev) in the case of initial data in the form of images (Descriptive approach to the analysis and understanding of images (DA)).

Gurevich introduced DIAs in the framework of the DA and continues to develop them in collaboration with his pupils [3,4,5,6,7,8,9]. In order to construct a DIA, it is necessary to select the operations and operands of the algebra. Some transformations in image processing, analysis and recognition can formally be used for mathematical description of the algorithm using DIAs, however, they have no physical meaning specific to image processing and analysis. At the same time, the practical applicability of DIAs is determined by the practical applicability and realizability of operations by which DIAs are constructed.

In our case, we are talking mainly about algebraic interpretation, since DIAs represent an algebraic language for the mathematical description of procedures of processing, analyzing, recognizing, and understanding images using digital image transformation operations and their representations and models.

These procedures are formed and implemented as descriptive algorithmic schemes (DASs) [4], which are correct (valid) expressions of the DIA language. The latter are constructed from image processing and transformation operations and other mathematical operations included in the corresponding DIA ring.

The mathematical and functional (content/semantic) properties of DIA operations are of considerable interest for optimizing the selection and implementation of image processing and analysis procedures and for constructing specialized DAS libraries.

The choice and optimization of operations included in the DAS are essentially related to the specifics of images as a means of representation, carriers, and sources of information. The functional interpretation of image transformation operations should ensure the establishment of a relationship between the image analysis task and the DAS specialized for its solution. In essence, this kind of interpretation is reduced to establishing a correspondence between the content division of the decision process into stages and the mathematical operations of the DAS ensuring the realization of these stages.

This means that the efficiency of model synthesis and recognition processes can be achieved by the choice of “content” (function) of image transformation operations, based on what image representation needs to be obtained with the next transformation. Such a choice, in turn, should be based both on the analysis of the mathematical characteristics of the operation and on the analysis of its functional purpose, in other words, the semantic aspects of the operation, i.e., its content, identification of a “physical equivalent,” and its underlying functional heuristics.

Since not all mathematical operations have a direct physical equivalent with respect to the construction of effective DASs for image analysis, there is the problem of interpreting operations for filling the DAS. Research into this problem leads to the selection and study of interpretation domains of DIA operations.

Thus, interpretation is considered as a transition from a meaningful description of the operation to its mathematical or algorithmic realization. As a result, the practical applicability of operations is revealed in the context of the more general concept of interpretability.

The following sections of the article present results related to the interpretability of DIA operations and examples of domains of interpretability for certain types of operations.

The article consists of the Introduction, three sections, the Conclusion, and References.

In Sect. 2 “Descriptive Image Algebras with One Ring”, the main specifics of DIAs with one ring are determined, from which the interpretability of operations is formalized and specified.

Section 3, “Types of Interpretability of Operations of Descriptive Image Algebras with One Ring”, describes the method and tools for formalizing the types of interpretability of image analysis and processing operations.

To characterize the interpretability of DIA operations, the following concepts are introduced: (1) physical meaning of the operation, (2) physical interpretability in the context of image analysis and processing, (3) visual interpretability in the context of image analysis and processing, (4) weak physical interpretability, (5) strong physical interpretability.

Section 4, “Examples of Interpretability of Descriptive Image Algebra Operations”, provides 4 examples of operands with operations, for which the interpretability is studied.

2 Descriptive Image Algebra with One Ring

In this section, let us briefly recall the basic properties of DIAs.

The algebraization of pattern recognition and image analysis was devoted to creating a universal language for the uniform description of images and transformations over them. In [3, 5], the algebraization stages of pattern recognition and image analysis are described in detail and the basic concepts for defining DIAs and DIA with one ring (DIA1R) are introduced. The most significant results of the initial stage of pattern recognition algebraization were Zhuravlev’s algebras of algorithms [12] and Grenander’s image theory [1]; in image analysis, Sternberg’s image algebra [10] and Ritter’s standard image algebra [11]. In common sense, by image algebra, we mean a mathematical theory describing image transformations and analysis in continuous and discrete domains [11].

The classical algebra was developed to generalize operations on numbers; however, direct application of an algebra to information in the form of images is not possible for all problems, and a simple interpretation of the results is not always admissible. There are many natural image transformations that are easily interpreted from the user’s viewpoint (e.g., rotation, compression, stretching, color inversion), which are difficult to imagine using standard algebraic operations. It becomes necessary to combine the algebraic apparatus and the set of image analysis and processing transformations.

One of the fundamental features of the algebraic approach is the representation of recognition algorithms in the form of algebraic combinations over a certain basis of algorithms. Another necessary prerequisite is algebraization of the representations of the input information of the algorithms, which in its capabilities is comparable to algebraization of the representations of the algorithms proper in the algebraic approach.

Correspondence of the algebraization of representations of algorithms and information is ensured by DA methods. These methods are designed to solve problems associated with obtaining formal descriptions of images as analysis and recognition objects and with the synthesis of procedures for their recognition by studying the internal structure, structure, and content of an image as a result of the generating operations by which the image can be constructed from primitive elements and objects detected in the image at various stages of its analysis [5, 9].

DIAs [3, 6] allow the use of image transformation procedures not only as DIA operations, but also as operands for constructing combinations of basic models of transformation procedures.

Definition 1

[3, 6]. An algebra is called a descriptive image algebra if its operands are either representations and models of images (as well, both the image itself and the set of values and characteristics associated with the image can be selected as a model), or operations on images, or simultaneously both.

In order to ensure compliance of the DIA with the requirements that must be met by the mathematical object “algebra,” it is necessary to introduce restrictions on the basic DIA operations.

The main research into DIAs was aimed at studying DIA1R (see Definition 2), which is by definition a classical algebra with nonclassical operands.

The subsequent specifics of DIAs are determined by the properties of the algebras.

Definition 2

[3, 6]. The ring, which is a finite-dimensional vector space over some field, is a DIA1R if its operands are either representations and models of images, or operations on images and their representations and models.

The ultimate goal in studying DIA1R is to obtain sets of complete systems of operands and DIA operations to describe image analysis tasks. The use of the algebra concept in defining a DIA1R in a strictly classical sense is governed by the fact that in this case, it becomes possible to distinguish the basic DIA operations for various types of operands. The interpretability of DIA1R operations for different types of operands was studied in [8]. In this paper, we present additional examples of strongly and weakly interpretable DIA1R operations.

3 Types of Interpretability of Operations of Descriptive Image Algebras

A problem arises in constructing a DAS for solving applied image analysis and recognition problems: the applicability of some classes of DIA to describe the corresponding problem [4]. Evaluating the applicability of the DIA leads to the problem of interpretability of DIA operations. The formulation of the problem and initial results are presented in [7, 8].

Recall [5] that, according to the DA, the source image in recognition tasks is called an ordered set of recorded initial spatial and contextual data, reflecting the form (form and state) of objects, events, and processes of the depicted scene and allowing application of transformations that produce an image convenient for recognition.

Definition 3

[3]. Physical meaning of the operation means a content description of the process of transforming the source image(s) into the final image(s), or the description of putting a certain set of characteristics into correspondence with the source image.

In order to preserve the logic of consideration below, let us recall some notions of the DA associated with description of the image processing and analysis process and leading to the definitions of model/image representation [5, 9].

In image processing and analysis, a certain system of transformations is applied to the source image, ensuring a successive change of “phase states” of the transformed image corresponding to the degree of its current “formalization.” The set of valid image representations is defined as the set of phase states of the image.

The system of transformations is given by the DAS image representation (DASIR), written according to DA concepts using DIAs. DASIRs reflect methods of sequential and/or parallel application of transformations from a set of transformations to the initial information from the initial data space. The set of admissible DASIRs is defined as the set of phase states of the DASIR.

To ensure the possibility of applying recognition algorithms to the constructed formal image descriptions, it is necessary to use the constructed DASIRs (to establish specific transformations from fixed DIAs and the parameters included in the transformation schemes) and to apply the implemented schemes to the initial data, i.e., construct image representations and models. In the DA, an image model is a formal (symbolic) description of an image that allows recognition algorithms to be applied to it. An image representation is any element of the set of states of the image in the image formalization space, with the exception of the objects “image model” and “image realization.”

A more detailed description of the image formalization space, including both the image phase states and the DASIR phase states, is given in [6].

Definition 4

[3]. An operation on an image(s) or fragments thereof, or on a model(s) of an image(s), or the representation(s) of an image(s) is called a physically interpretable operation in the context of image analysis and recognition if

  1. (1)

    the result of its use is an image or fragments thereof;

  2. (2)

    the result of its application is an image representation or image model that can be used to reconstruct semantically significant geometric objects, brightness characteristics, and configurations formed due to regular repetitions of geometric objects and brightness characteristics of the source image;

  3. (3)

    the result of its application is a characteristic(s) of the image(s), which can be unambiguously compared to the properties of geometric objects, brightness characteristics, or configurations formed due to regular repetitions of geometric objects and brightness characteristics of the source image.

Definition 5

[3]. The operation on some objects is called visually interpretable in the context of image analysis and recognition if as a result of the operation, an image(s) is obtained with which it is possible to reconstruct a one-to-one correspondence between semantically significant geometric objects, brightness characteristics, and configurations formed due to regular repetitions of geometric objects and brightness characteristics in the resultant image(s) and in source objects.

Statement 1

[3]. A visually interpretable operation is always a physically interpretable operation.

Corollary

[3]. If the operation is not a physically interpretable operation, then this operation is also not visually interpretable.

Physical interpretability can be distinguished in a strong and weak sense.

Definition 6

[3]. An operation is called strongly physically interpretable if it is also visually interpretable.

Definition 7

[3]. An operation is called weakly physically interpretable if it is physically interpretable, but not visually interpretable.

Visually interpretable operations include, e.g., image rotation, image shift, image contrast enhancement, image brightness enhancement, image noise reduction, image smoothing, image contour selection, and other image processing operations. An example of visually interpretable operations can also be image-constructing operations according to a certain specified rule from a set of original objects, e.g., image reconstruction from equations that define the image type.

Physically interpretable operations include certain operations of constructing image representations and models and such operations with images as calculation of the image histogram or the values of the image’s statistical characteristics.

Statement 2

[3]. An operation is physically uninterpretable in the context of image analysis and recognition if

  1. (1)

    its operands are not images, image models, image representations, or image fragments;

  2. (2)

    as a result of application of an operation to the image(s), an image model(s) is constructed, with which it is not possible to reconstruct semantically significant geometric objects, brightness characteristics, or configurations arising due to regular repetition of geometric objects and brightness characteristics of the source image;

  3. (3)

    as a result of application of an image operation, characteristics are calculated that cannot be unambiguously compared with the properties of geometric objects, brightness characteristics, or configurations arising due to regular repetition of geometric objects and brightness characteristics of the source image;

  4. (4)

    an operation is not applicable to images, image models, image representations, or image fragments.

4 Examples of Interpretability of Descriptive Image Algebra Operations

This section provides examples of the DIA listed in Table 1.

Table 1. Examples of DIA1R.
Full size table

Example 1. A DIA1R Over Image Algebra Operations

Let us demonstrate an example of a descriptive algebra that is an analog of the recognition algorithm algebra. Let be \( I \) an image, F be the field of real numbers, and elements of ring R be the operations of the image algebra [11]. Let \( r_{1} ,r_{2} \in R, \alpha \in F \).

The following operations are introduced in ring R:

$$ \left( {r_{1} + r_{2} } \right)\left( I \right) = r_{1} \left( I \right) + r_{2} \left( I \right) $$
(1)

Physical meaning of the operation: addition of the elements of the algebra is understood as addition of the results of applying the image algebra operations to the image.

$$ \left( {r_{1} \cdot r_{2} } \right)\left( I \right) = r_{1} \left( I \right) \cdot r_{2} \left( I \right) $$
(2)

Physical meaning of the operation: multiplication of elements of the algebra is understood as multiplication of the results of applying the image algebra operations to the image.

$$ \left( {\alpha r_{1} } \right)\left( I \right) = \alpha r_{1} \left( I \right) $$
(3)

Physical meaning of the operation: multiplication of an element of the algebra by an element of the field of real numbers is understood as multiplication of an element of the field of real numbers and the result of application of the operation of the image algebra to an image.

As ring elements, it is possible to choose both operations that transfer images to other images and operations that construct certain image models for images, e.g., numerical estimates of their characteristics.

In the first case, the algebra can be considered an apparatus for constructing a chain of image transformations necessary for constructing the final procedural representation/model of an image [5, 9]. In the second case, the algebra is a convenient tool for representing algorithms as a composite of algorithms from a given basis with given operations over them.

Statement 3.

The operations of addition (1) and multiplication (2) of two image algebra operations applied to the image are weakly physically interpretable operations.

Proof

  1. 1.

    Physical interpretability of operations: By Definition 5, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operations: these operations are not visually interpretable, since their application to arbitrary images obtained before this during application to the image of image algebra operations leads to an unpredictable visual result (Definition 5).

  3. 3.

    By Definition 6, a physically interpretable, but not visually interpretable operation is a weakly physically interpretable operation.

Q.E.D.

Statement 4.

The operation of multiplication of the image algebra applied to the image by field element (3) is a strongly physically interpretable operation.

Proof

  1. 1.

    Physical interpretability of operation: By Definition 4, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operation: the operation is visually interpretable (Definition 5).

  3. 3.

    By Definition 7, a physically interpretable and visually interpretable operation is a strongly physically interpretable operation.

Q.E.D.

Example 2. A DIA1R Over Numerical Estimate Construction Operations

Let I be an image, F be the field of real numbers, and the elements of ring R be numerical estimate construction operations. Each operation is represented by a function f, which relates an image to a number or a set of numbers—a feature vector. Let be \( r_{1} ,r_{2} \in R, \alpha \in F \), where \( r_{i} \left( I \right) = f_{i} \left( I \right) \) (i = 1, 2) (in this case \( f_{1} \left( I \right) \) and \( f_{2} \left( I \right) \) are vectors of the same dimensionality or real numbers). Let the values of all features be considered on the segment [0, 1] of the real number axis. The dimensionality of the feature vector is fixed; unknown feature values are replaced by a value of 0.5.

We introduce the following operations in the ring:

$$ \left( {r_{1} + r_{2} } \right)\left( I \right) = f_{1} \left( I \right) + f_{2} \left( I \right) - f_{1} \left( I \right) \cdot f_{2} \left( I \right) $$
(4)

Physical meaning of the operation: addition of the elements of the algebra is understood as addition of the results of applying the corresponding functions for computing estimates to the image minus multiplication of the results of applying the corresponding functions to the image.

$$ \left( {r_{1} \cdot r_{2} } \right)\left( I \right) = f_{1} \left( I \right) \cdot f_{2} \left( I \right) $$
(5)

Physical meaning of the operation: multiplication of the elements of the algebra is understood as multiplication of the results of applying to the image the corresponding functions for computing estimates.

$$ \left( {\alpha \cdot r_{1} } \right)\left( I \right) = \alpha \cdot f_{1} \left( I \right) $$
(6)

Physical meaning of the operation: multiplication of an element of the algebra by a field element is understood as multiplication of the field element by the result of applying to the image the corresponding function of computing estimates.

Statement 5.

The operations of addition (4) and multiplication (5) of two for numerical estimate construction operations applied to the image are weakly physically interpretable.

Proof

  1. 1.

    Physical interpretability of operations: By Definition 4, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operations: these operations are not visually interpretable, since their application to the functions of calculating the estimates applied to the image leads to a new estimate that does not have a visual relationship with the image (Definition 5).

  3. 3.

    By Definition 6, a physically interpretable, but not visually interpretable operation is a weakly physically interpretable operation.

Q.E.D.

Statement 6.

The operation of multiplication of the numerical estimate construction operation by field element (6) is strongly physically interpretable.

Proof

  1. 1.

    Physical interpretability of operations: By Definition 4, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operation: the operation is visually interpretable (Definition 5).

  3. 3.

    By Definition 7, a physically interpretable and visually interpretable operation is a strongly physically interpretable operation.

Q.E.D.

Example 3. A DIA1R Over Rotation and Scaling Operations.

Let I be an image, F be the field of real numbers, and elements of ring R be rotation and scaling operations represented as the pair \( r_{i} = \left( {s_{i} , t_{i} } \right), i = 1,2, \ldots \) Let \( r_{1} ,r_{2} \in R, \alpha \in F \).

We introduce the following operations in the ring:

$$ \left( {r_{1} + r_{2} } \right)\left( I \right) = \left( {s_{i} + s_{j} ,t_{i} + t_{j} } \right)\left( I \right) $$
(7)

Physical meaning of the operation: the addition of the elements of the algebra is understood as the pair of the total angle of rotation and total scale of the image.

$$ \left( {r_{1} \cdot r_{2} } \right)\left( I \right) = \left( {0, t_{1} \cdot t_{2} } \right)\left( I \right) $$
(8)

Physical meaning of the operation: multiplication of the elements of the algebra is understood as the pair with an angle of rotation equal to 0 and with multiplication of the image scales.

$$ \left( {\alpha \cdot r_{1} } \right)\left( I \right) = \left( {s_{i} ,\alpha \cdot t_{i} } \right)\left( I \right) $$
(9)

Physical meaning of the operation: multiplication of an element of the algebra by a field element is understood as the pair of the initial angle of rotation and image scale multiplied by \( \alpha \).

Statement 5.

The operations of addition (7) and multiplication (8) of the two rotation and scaling operations applied to the image, as well as the operation of multiplication of the rotation and scaling by field element (9) are strongly physically interpretable.

Proof

  1. 1.

    Physical interpretability of operations: By Definition 4, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operations: these operations are visually interpretable (Definition 5).

  3. 3.

    By Definition 7, a physically interpretable and visually interpretable operation is a strongly physically interpretable operation.

Q.E.D.

Example 4. A DIA1R Over Standard Algebraic Operations

Let I be an image, F be the field of real numbers, and elements of the ring R be standard algebraic operations. Let \( r_{1} ,r_{2} \in R, \alpha \in F \). \( r_{i} \left( I \right) \) corresponds to a standard algebraic operation over a pair (I, Ti), where Ti is some fixed object (standard, model), \( i \, = \, 1,2, \ldots \).

Some of G. Ritter’s image algebra operations are introduced in ring R so that the properties of the algebra are fulfilled (e.g., the operations of pointwise addition and multiplication of two images, or the operations of pointwise taking of the maximum and minimum of two images):

$$ \left( {r_{1} + r_{2} } \right)\left( I \right) = r_{1} \left( I \right) \oplus r_{2} \left( I \right) $$
(10)

Physical meaning of the operation: addition of the elements of the algebra is understood as application of the operation of addition of two images obtained after application of standard algebraic operations to the original image and the given template.

$$ \left( {r_{1} \cdot r_{2} } \right)\left( I \right) = r_{1} \left( I \right) \odot r_{2} \left( I \right) $$
(11)

Physical meaning of the operation: multiplication of elements of the algebra is understood as application of the operation of multiplication of two images obtained after application of standard algebraic operations to the original image and a given template.

$$ \left( {\alpha r_{1} } \right)\left( I \right) = \alpha r_{1} \left( I \right) $$
(12)

Physical meaning of the operation: multiplication of an element of the algebra by an element of the real number field is understood as pointwise multiplication of an element of the real number field and the result of application of a standard algebraic operation to the original image.

Statement 5.

The operations of addition (10) and multiplication (11) of two standard algebraic operations applied to the image and the given templates, as well as the operation of multiplication of a standard algebraic operation applied to the image and the given template by field element (12), are weakly physically interpretable.

Proof

  1. 1.

    Physical interpretability of operations: By Definition 4, an operation is physically interpretable if its application results in an image, or an image representation or image model, or characteristics of the source image.

  2. 2.

    Visual interpretability of operations: these operations are not visually interpretable, since their application to standard algebraic operations previously applied to the image and given templates generally leads to unpredictable images (Definition 5).

  3. 3.

    By Definition 6, a physically interpretable, but not visually interpretable operation is a weakly physically interpretable operation.

Q.E.D.

5 Conclusion

This paper continues to study the formal aspects of interpretability for all major DIA operations. The main types of interpretability are defined, and examples of interpretability of operations of the DIA1R are given.

Also of interest is the formulation of the problem of the formalization of interpretability of sets and bases of the standard image-processing operations of DIA1R. This problem is apparently related to the concept of image equivalence [2].

Note that sometimes the interpretation of algorithmic procedures for image processing and analysis is understood as the construction of descriptive algorithmic schemes in the language of DIA1R for solving applied problems [4, 8].

The results of this work can be used to study the interpretability of specialized DIA1R operations designed to solve problems of image analysis.