Keywords

1 Introduction

Based on physiological abilities, a person receives most of the information based on the visual apparatus. Therefore, there is a legitimate trend towards increasing attention, attractiveness, and informativeness of graphic information. Moreover, the ability to visualize the image in a three-dimensional format is important. Here it is necessary to highlight the volumetric display [1,2,3,4,5,6,7,8,9,10], in which the volumetric image is created by volumetric pixels or voxels. The autostereoscopic way of displaying graphical information implies the ability to observe an object from various angles, which is preferred for the observer [11,12,13,14,15,16,17,18,19,20,21,22]. Based on this, it can be concluded that technologies aimed at reproducing a three-dimensional image are currently quite promising.

The team of authors based on the combined use of optical and computer processing proposed [23] developed several autostereoscopic displays that differ in design solutions. The model range of volumetric displays proposed based on an analysis of user needs currently includes 5 modifications.

Below is a brief description of them and examples of the 4th and 5th modifications (Figs. 1 and 2):

Fig. 1
figure 1

Type 4 display

Full size image
Fig. 2
figure 2

Type 5 display

Full size image
  1. 1

    A universal mobile nozzle for creating a three-dimensional display without moving parts using multilayer inclined projecting faces.

  2. 2

    A universal mobile nozzle for creating a three-dimensional display without moving parts using parallax barriers and a video sequence from a set of interlaced stereo pairs.

  3. 3

    Volumetric display without moving parts using parallax barriers, interlaced stereo pairs projected on the inclined faces of the pyramid.

  4. 4

    Volumetric circular view display with a rotating working area and an array of projectors (Fig. 1).

  5. 5

    Volumetric display without moving parts using a horizontal stereo pair and an optical system from a lens necklace (Fig. 2).

In the proposed model range, the above-combined principle of forming a three-dimensional image is used, which is described by the following scheme [24] (Fig. 3).

Fig. 3
figure 3

The sequence of transformations of the original image when forming a three-dimensional image. 1 original image of objects 3D models, 2 Processing formation of primary images, 3 transferring images to input devices, 4 image sync, 5 image transfer to the optical system, 6 formed volumetric image

Full size image

Under this scheme, a composite image P′ is formed from the primary images P by the output device. At the same time, for the correct connection of the composite image elements, synchronization is necessary. Next, we get an image Q, which is synchronized both in time and in space. This image is transmitted to the optical system to form the resulting volumetric image Y. This image is perceived directly by the observer in the form of image Y′. In this case, it is necessary to take into account its psychophysiological features Z.

According to the scheme in Fig. 3, the set of characteristics of two-dimensional images in a certain way affects the final volumetric image. The values of the input and output variables of the system are determined based on these characteristics. It is necessary to take into account certain restrictions that are caused by the use of various technical devices. Moreover, restrictions affect the final volumetric image and also determine the quality of the human perception of the volumetric image.

To solve this so-called direct problem associated with determining the values of Y′ from the given characteristics of the input image X and the parameters of the procedure for generating a three-dimensional image perceived by the observer, we proposed an appropriate mathematical model based on the use of status functions [25].

When designing an autostereoscopic display from the proposed model range and possible others, it is necessary to solve the so-called inverse problem. In this case, according to the required characteristics of the output volumetric image, it is necessary to determine the values of the variables that describe the input image and the process of forming the volumetric image using the devices used that implement the process in Fig. 3. In the general case, inverse problems are known to be incorrectly posed. This is true in our case since the uniqueness and existence of a solution cannot be guaranteed.

This article is devoted to the solution to this urgent scientific and technical problem.

2 Setting Goals and Objectives of the Study

The quality of the output volumetric image and the level of its perception by a person depends on the quality of the input image, the internal structure of the autostereoscopic display, and individual perception. Therefore, it is important to determine the permissible set of values of the input variables of the system and the requirements for the weekend, using the information on the user’s preferences for the quality of the final image or scene.

The study aims to synthesize the characteristics of the parameters of autostereoscopic displays from a given set with the required indicators of the quality of formation of the output volumetric image, taking into account the parameters of the input images and the property of devices that implement the process of forming the volumetric image.

To achieve this goal it is required to solve the following tasks:

  1. 1

    To determine the most informative variables, as well as their interrelation in the formation of volumetric images by displays of a given model series. Highlight the variables used in design and user decision-making processes.

  2. 2

    To develop the architecture of a decision support system for the design of autostereoscopic displays of a given model range.

  3. 3

    To test the proposed solutions for the automation of the process of designing autostereoscopic displays.

The result of the research is a software package to support decision-making by the designer in the design of autostereoscopic displays, as well as for users to select and adjust the characteristics of displays of a given model range.

The proposed decision support system (DSS) should solve the direct and inverse problem in the design of autostereoscopic displays. The direct task is to calculate the values of the output characteristics of the generated volumetric image from the given input. The inverse problem is related to determining the requirements for the values of the parameters of the input image and the design of the display at given values of the output volumetric images.

Next, we consider the results of solving the problem of determining the most informative variables and parameters that determine the quality of a three-dimensional image.

3 Characterization of Variables and Parameters of the Process of Forming Three-Dimensional Images that Determine the Quality of the Three-Dimensional Image

To achieve the required quality of the volumetric image at the output of the volumetric display, it is necessary, as mentioned above, to solve the complex problem of choosing a technical solution, design features, and also the quality of the input image. Therefore, to automate the process of selecting the required characteristics of volumetric displays, it is advisable to use software that can indicate possible solutions to the problem, while leaving the right to choose the final option to the user and designer.

As a tool for solving the problem, it is proposed to use a decision support system to determine the parameters of volumetric displays, in particular, the composition of the optical system and the values of the quantitative characteristics of individual elements, which in the future will be useful for structural engineers, designers when developing or testing the display design, capable of forming an image of a given quality. Also, the obtained values can be used in software image processing. The generated values for the change in the variables of the reference frames that are proposed by the DSS can directly affect the quantitative and qualitative characteristics of the final image, bringing them closer to the desired value.

To create a DSS, a formalized description of the subject area of volumetric displays was carried out, and procedures for determining the output characteristics of visualization processes were developed. Also, databases of parameters for displays of characteristics of primary images, visualization parameters were formed, and relationships were determined (Table 1).

Table 1 The adjacency matrix of the relationships between the parameters affecting the quality of the generated volumetric image
Full size table

Quality of output three-dimensional image depends on the degree of technical realization of variable characteristics of an input image by output devices, the possible range of variable variation, as well as value set by a user when setting values of input parameters of image quality.

The optical characteristics of the visual apparatus of a person and the assessment of his level of perception do not affect the technical characteristics of the system and image. However, alterations performed in reverse have a significant effect on optical characteristics (the so-called inverse problem, which is usually the number of incorrect ones).

The perception of the object is also influenced by psychological factors and physiological features of the person. Therefore, another aspect of the development of a three-dimensional imaging technique is the need to test it on different groups of recipients.

Several experiments are proving this aspect, including:

  1. 1

    theory of Gregory’s visual sentence [26];

  2. 2

    phenomenon of binocular rivalry [27];

  3. 3

    perception of the essence of the scene according to Castellano and Hendersen;

  4. 4

    cognitive, figurative-structured visual perception (COSV).

In addition to the described, there are specific and currently not identified by medicine psychological factors. Several diseases that affect the human brain and visual apparatus should also be considered:

  1. 1

    astigmatism;

  2. 2

    refraction;

  3. 3

    visual agnosia.

Thus, it is advisable to use a relatively large sample in testing, taking into account the various states of the subjects, both physical and psychological.

Besides, the estimated characteristics of the identified parameters are introduced based on the processing of expert estimates (Table 2).

Table 2 Weighting factors of the characteristics of the output image
Full size table

To use volumetric images in various subject areas in cyber-physical systems [28,29,30,31,32] when constructing a mathematical description of the perception of a volumetric image, it is advisable to use the so-called status functions [25]. The above characteristics are used in the construction of DSS, the architecture of which is described below.

4 Development of the Architecture of a Decision Support System for Building Volumetric Displays

The development of autostereoscopic displays using a combined method and certain technical solutions to achieve preset values of the perceived quality in a volumetric manner is an important scientific and technical task. Therefore, it is advisable to automate the design of a promising lineup of autostereoscopic displays. To do this, it is necessary to develop and apply the required software.

The proposed DSM is necessary for determining the values of the parameters of autostereoscopic displays, as well as the process of forming three-dimensional images based on primary images. This information is very useful for designers in the development of design processes, hardware, and software for the production of autostereoscopic displays with the required values of the quality indicators of the perception of output volumetric images.

Functionally, DSM should be based on a formalized description of the subject area of autostereoscopic displays. Also, procedures should be developed for calculating the values of the output characteristics during visualization (direct task). Database of characteristics of autostereoscopic displays, parameters of primary images, as well as volumetric visualization is required. Further, modules are required for selecting parameters of autostereoscopic displays to provide the specified quality of volumetric visualization (inverse problem).

In connection with the foregoing, DSM architecture is proposed (see Fig. 4), which includes the necessary components. Among them, information support, which contains the following resources:

Fig. 4
figure 4

Architecture DSM for the development of autostereoscopic

Full size image

  • Database (DB) of parameters of autostereoscopic displays;

  • Knowledge Base (KB) of the subject area of autostereoscopic displays;

  • DB of status functions;

  • KB for decision support in the field of autostereoscopic displays;

  • DB of visualization parameters;

  • DB of characteristics of primary images.

  • The decision support subsystem for input image and configuration selection includes the following modules):

  • type selection from the range of autostereoscopic displays;

  • determination of input image parameter values (contrast, brightness, etc.) for autostereoscopic displays of a given model range;

  • determination of parameter values that describe the dynamics of images (frame change, resizing, moving, rotating,) for autostereoscopic displays of a given model range;

  • determination of the parameter values of the optical and hardware components of autostereoscopic displays;

  • output of the resulting information in the form of tables.

The functioning of DSS is carried out under the control of the control module, which organizes the necessary interaction of the modules to support decision-making and information support, as well as two groups of users. These include the administrator and operator, who is the user, or a testing knowledge engineer.

The administrator manages the differentiation of access levels for various categories of users, can enter changes in the source code, algorithm, as well as change and adjust data in the database and database, based on confirmed feedback from the user and experimental data from the engineer.

The user in a step-by-step mode carries out the design process, based on the answers to the questions that determine the most important criteria for the volumetric display and the input image, as a result, the display type, the main parameters of the primary images are determined, and the engineer, in turn, based on the received data, can conduct mathematical modeling of the trace beam or visualize the process of building the image on pre-prepared software, to assess the quality of the resulting three-dimensional image and evaluate the effectiveness of the selection of characteristics performed by DSS. During the operation of the system, the modules (subprograms) are called up to determine the appropriate type of display, to calculate the values of static (brightness, contrast, etc.) characteristics, characteristics that describe the dynamics (rotation, movement, change of frames) on the input image, and also the module for calculating variable values hardware and optical component for displays of the proposed type.

It is proposed to use cloud storage, the level of access to which is also differentiated for different groups, to store the database, database, user survey results, and also the values obtained from the system.

The advantage of such a system is the multidimensionality of the direction of the received data, i.e. access to the results is available to all user groups, which simultaneously allows you to:

  1. 1

    changing the input image by the user;

  2. 2

    creating a display design by a design engineer;

  3. 3

    verification of the effectiveness of the proposed system utilizing mathematical modeling by an engineer-developer;

  4. 4

    editing algorithms and methods for presenting information by a programmer;

  5. 5

    monitoring the functioning of the system by the administrator.

5 Implementation of a Decision Support System for the Design of Autostereoscopic Displays

When implementing the proposed DSS of design, a scheme was developed to identify user and design preferences (requirements). This scheme is implemented in the corresponding module written in the C# programming language. Using this procedure allows you to clearly define the requirements of the designer, user when designing autostereoscopic displays or setting parameters from a given model range of volumetric displays.

To solve the inverse design problem associated with achieving the required quality of the output image based on the selection of the characteristics of the input images, the visualization process, the knowledge of experts is used. Their formalization is carried out based on the production rules “IF THAT”, which is implemented using the Prolog programming language of artificial intelligence.

The following are fragments of the procedure for identifying requirements for the characteristics of autostereoscopic displays using the appropriate module and screenshot of the result.

When functioning DSS first you need to determine the list of questions about the quality of the output image. Based on them, direct or inverse problems are further solved. Questions for increasing information content are presented in a visual form.

So, for example, the second question to the user is about the desired viewing angle from 0 to 360°. Displays of the 1st, 2nd type of the model range are limited by a viewing angle of up to 180° of one projecting plane. A type 3 display can be realized with a viewing angle of 90, 270, or 360°, when selecting one of 3 or 4 projection planes, respectively. Displays of the 4th and 5th type can have any of the possible ranges, however, with an increase in the viewing angle, the overall dimensions of the product significantly increase.

The last question allows the user to determine the diagonal of the screen for building a 3D image with the desired dimensions. You can enter the length, depth, and width of the image in centimeters. To determine the diagonal of the screen, the length and width of the image are sufficient. The diagonal is calculated and then converted to inches and rounded to the nearest whole number to approximate the actual size of the displays. Further, the 11th question also allows you to determine the area of the three-dimensional image, which also introduces some restrictions on the use of a particular type of display. Due to the nature of the optical system, the 4th type display does not allow the image to be formed over 1000 cm3, and the 5th type display is limited to 8000 cm3.

Depending on the list of displays available for playback from the model range, DSS displays information of a recommending nature for the user and developer (Fig. 6) per the operation algorithm (Fig. 5).

Fig. 5
figure 5

Algorithm for determining the characteristics of autostereoscopic displays based on user and design requirements

Full size image
Fig. 6
figure 6

An example of the result of the work of DSS

Full size image

The following characteristics are defined for the user:

  • brightness level;

  • contrast level;

  • minimum fps value;

  • image resolution.

For a design engineer, a set of characteristics is given below:

  • number of lenses;

  • LPI value;

  • diagonal display;

  • number of mirrors.

6 Conclusion

The article discusses the development of promising auto-stereoscopic displays, allowing them to form three-dimensional images visible from different sides without additional devices (glasses, etc.). The characteristic of the lineup of autostereoscopic displays proposed by the authors is developed based on the patent [23]. To solve decision support, the architecture of the corresponding system is proposed, which allows you to automate the solution of the direct and inverse problems that arise during the design of volumetric displays and their use. A description of the developed algorithm for determining the values of the characteristics of the designed autostereoscopic display from the proposed model range is given. An example of testing an algorithm implemented in the C# programming language is presented. To solve direct and inverse problems in the proposed hybrid expert decision support system, formalization of expert knowledge using the Prolog artificial intelligence programming language is proposed. It is advisable to use the obtained results in cyber-physical systems for designing systems using volumetric visualization tools.