Technology of Integrated Application of Classical Decision Making Criteria for Risk-Uncertainty Assessment of Group Systems of Preferences of Air Traffic Controllers on Error’s Dangers

Abstract

The study of individual systems of air traffic controllers’ (ATC) preferences on the dangers of characteristic errors has a positive proactive character. Group systems of preferences (GSP) reveal features of functioning of separate societies – ATC’s shifts. Individual systems of preferences m = 37 tested air traffic controllers were built. The implementation of a multi-step technology for detecting and rejecting marginal thoughts has led to a statistically consistent GSP: Kendall's concordance coefficient is W = 0.700 and is statistically significant at an unusually high level of significance for human factor research α = 1%. A decision matrix has been formed – a “cost matrix”, for the solution of which the methodology of application of classical decision-making criteria by Wald (W), Savage (S), Bayes-Laplace (B-L), Hurwitz (HW) has been implemented. Empirical preferences coincide: the values of Spearman's rank correlation coefficients are equal to $R_{S}^{B - L - W/S}$ = 0.8922, $R_{S}^{B - L - HW}$ = 0.9263 and are statistically significant at the level of significance α = 1%. The values of the normalized risk index of indistinguishability of error risks in group systems of advantages are equal to: $R_{BL}^{*}$ = 0, $R_{HW}^{*}$ = 0.19⋅10^–2, $R_{W/S}^{*}$ = 5.58⋅10^–2. For the group as a whole $R_{g}^{*}$ = 0.52⋅10^–2.

1 Introduction

Today, air traffic controllers (ATC), together with flight crew members, are considered “front-line” and “last frontier” aviation operators (AO), as they have a direct, both positive and, statistically, mostly negative impact on ensuring proper level of flight safety (FS) [1, 2]. Therefore, the study of HF problems, especially proactive, and the practical implementation of their results, is a more important factor in preventing accidents. After all, during the last 60–70 years at least 2/3–3/43 of these events arose due to the negative impact of the human factor (HF) [3].

Purposeful and complex polyergatic control system “Flight crew – aircraft – environment – ATC” is humanistic, according to one of the founders of fuzzy mathematics [4]. Thus aviators also have the right to make mistakes [1–4 etc.]. Moreover, these errors should be considered in the context of decision-making (DM), as the professional activity of “front line” aviation operators (AO) is usually considered as a continuous chain of decisions that are made and implemented in explicit and implicit forms and under the influence of many different factors. In addition, the vast majority of accidents are the result of wrong decisions.

Recalling the well-known Latin proverb “Praemonitus, praemunitus” (warned, therefore, armed), it would be expedient to form in JSC “leading edge” skills of recognition, assessment of dangers, memorization, and, consequently, prevention of erroneous actions and decisions in professional activity.

That’s why, as the experience of research [5,6,7] shows, the identification of individual and group systems of advantages (SP) of “leading edge” AO on the indicators and characteristics of professional activity, in particular, on the dangers of characteristic errors that they may assume, performing operating procedures. It was found that the controllers of ATS, who were accidentally involved in the construction of such individual SP before training, made in its process a third fewer errors than those that were not covered by this procedure.

In the context of our research, the system of preferences (SP) will be understood as the representation of the ATS about the most and the least dangerous mistake, and hence – about the completely orderly series of mistakes that they can make in professional activities.

2 Analysis of Researches and Publications

Let us draw attention to the ICAO-recognized basic model of error management, proposed by Professor of the University of Texas, Dr. Robert Helmreich [8]. However, the model is focused on pilot’s error management and does not take into account the specifics of the professional activities of air traffic controllers (ATC’s).

Significant studies of ISP and GSP of air traffic controllers from Azerbaijan and Ukraine were conducted under the guidance of one of the co-authors. Based on world accidents and incidents statistics at ATC, ICAO recommendations, as well as personal experience of practical ATC, teaching and instructor work of the authors of this publication, a list of n = 21 characteristic errors was generated, which is currently the most complete and comprehensive coverage of the inappropriate actions of air traffic controllers: Er.₁ is the Violation of radiotelephony phraseology; Er.₂ is the Inconsistent entry of the aircraft into the zone of the adjacent ATC; Er.₃ is the Violation of longitudinal course time separation; Er.₄ is the Violation of time separation on reciprocal tracks; Er.₅ is the Violation of separation between aircrafts on crossing tracks; Er.₆ is the Address less ATC messaging; Er.₇ is the Error in determining of aircraft call sign; Er.₈ is the Error in aircraft identification; Er.₉ is the Misuse of ATC schedule; Er.₁₀ is the Absence of mark of the control transfer to the adjacent Air traffic control center in the strip; Er.₁₁ is the Absence of mark of the coordination of the entrance of the aircraft to the adjacent ATC area in the strip; Er.₁₂ is the Violation of coordinated geographic boundary by ATC; Er.₁₃ is the Violation of coordinated time of control transfer at FIR – boundary by ATC; Er.₁₄ is the Negligence in applying to the strip of the letter-digital information (the possibility of double interpretation); Er.₁₅ is the Non-economical ATC; Er.₁₆ is the Violation of shift handover procedures; Er.₁₇ is the Issued commands to change the altitude or direction of flight are not reflected on the strip; Er.₁₈ is the Attempt to control the aircraft under condition of TCAS system operation in the resolution advisory mode; Er.₁₉ is the Errors in entering information about aircraft into an automated system; Er.₂₀ is the Violation of emergency procedures; Er.₂₁ is the Violations of airspace use.

A multi-step procedure for detecting and weeding out marginal thoughts was implemented, classical DM criteria were used to detect GSP, an indicator for assessing the degree of their risk (indistinguishability of alternatives-errors) was introduced, and the Kemeny’s median was constructed as an optimized GSP that gives the most complete idea of true group wounds.

However, when establishing the ISP, the normative method of distribution of the total risk of errors was used, which led to a certain “coarsening” of rank assessments of the risk of errors in both the ISP and the GSP. The classical Hurwitz’s criterion was not used in the construction of the GSP.

Previously a differential method of detecting part of the total risk of errors was developed and implemented by authors, which led to obtaining more accurate ISPs, and hence GSPs. Which contributed to an increase in the consistency of errors (Kendall’s concordance coefficient) immediately by 1.92 times relative to its indicator calculated for GSP, which were obtained by generalization of ISP, obtained by the traditional method of normative distribution of the total risk of error. The multi-step technology of detection and elimination of marginal thoughts is realized. For the first time, the criteria of danger and frequency of adverse events proposed by ICAO were used for an integrative (holistic, aggregate) assessment of the level of undesirability of errors. However, neither the classical DM criteria nor the Kemeny’s median was used to obtain the GSP based on these results.

Therefore, taking into account the impact of HF on FS, as well as the results of analysis of ATC’s SP on the dangers of characteristic errors, the purpose of this publication is to build GSP using the classical criteria of DM, as well as assess their risk from the standpoint of distinguishing the dangers of orderly errors.

3 Forming and Solving a Matrix of Solutions

The methodology of applying the classical criteria of DM to solve applied technical and economic problems is well known. However, recommendations for taking into account the peculiarities of the impact of the HF on DM, especially in aviation systems with the use of classical criteria contains a limited number of works. Based on the above, consider the appropriate technology.

Thus, $m = 37$ of the respondents-ATCs arranged $n = 21$ of the characteristic errors, using the differential method of determining the comparative risk of errors. Using multi-step technology to detect and weed out marginal thoughts, this sample was reduced to m_A = 27 members who have a high level of intragroup agreement: the Kendall concordance coefficient is equal to $W_{{m_{A} }} = 0.7$ and is statistically plausible at an unusually high significance level for HF studies $\alpha = 1\%$. The aggregation of ISPs for members of this subgroup in GSP was carried out using a group decision strategy such as summation and averaging of ranks. The formal type is

$$ \begin{array}{*{20}c} {E_{18} \mathop \succ \limits_{{m_{A} }} E_{20} \mathop \succ \limits_{{m_{A} }} E_{5} \mathop \succ \limits_{{m_{A} }} E_{21} \mathop \succ \limits_{{m_{A} }} E_{4} \mathop \succ \limits_{{m_{A} }} E_{3} \mathop \succ \limits_{{m_{A} }} E_{8} \mathop \succ \limits_{{m_{A} }} E_{17} \mathop \succ \limits_{{m_{A} }} E_{13} \mathop \succ \limits_{{m_{A} }} E_{2} \mathop \succ \limits_{{m_{A} }} E_{16} \mathop \succ \limits_{{m_{A} }} } \\ {\mathop \succ \limits_{{m_{A} }} E_{19} \mathop \succ \limits_{{m_{A} }} E_{6} \mathop \succ \limits_{{m_{A} }} E_{12} \mathop \succ \limits_{{m_{A} }} E_{7} \mathop \succ \limits_{{m_{A} }} E_{1} \mathop \succ \limits_{{m_{A} }} E_{14} \mathop \succ \limits_{{m_{A} }} E_{11} \mathop \succ \limits_{{m_{A} }} E_{9} \mathop \succ \limits_{{m_{A} }} E_{10} \mathop \succ \limits_{{m_{A} }} E_{15} ,} \\ \end{array} $$

(1)

where $\mathop \succ \limits_{{m_{A} }}$ is the mark of the advantage of the danger of one error over another in the GSP (1), constructed by the differential method of comparing their dangers.

The ISPs of the members of the m_A subgroup form a decision matrix (Table 1), which is the so-called “loss matrix”, because the smaller the absolute value of the rank of the i-th error in the ISP of the j-th expert (r_ij), the more dangerous it is. The solution of the decision matrix can be done with the help of classical DM criteria.Abraham Wald's criterion is considered a criterion of extreme pessimism (caution), because its application contributes to a guaranteed result. The approach based on the main principles of systems analysis, known as “removal of uncertainty”. According to him, the best solution (the most dangerous mistake) is from the analysis of Table 1 as follows:

$$ Z_{W} = \mathop {\min }\limits_{i} \;r_{ik} = \mathop {\min }\limits_{i} \;\mathop {\max }\limits_{j} r_{ij} . $$

(2)

According to Table 1 and Eqs. (1)–(5), we conclude that the ATCs GSP on the dangers of characteristic errors were the same when applying the Wald criterion and the Savage criterion. Their formal description is as follows:

$$ \begin{array}{*{20}c} {E_{4} \mathop \succ \limits_{W} E_{20} \mathop \succ \limits_{W} E_{5} \mathop \approx \limits_{W} E_{18} \mathop \succ \limits_{W} E_{3} \mathop \succ \limits_{W} E_{21} \mathop \succ \limits_{W} E_{2} \mathop \succ \limits_{W} E_{12} \mathop \approx \limits_{W} E_{17} \mathop \succ \limits_{W} E_{13} \mathop \succ \limits_{W} E_{8} \mathop \succ \limits_{W} } \\ {\mathop \succ \limits_{W} E_{1} \mathop \succ \limits_{W} E_{7} \mathop \succ \limits_{W} E_{6} \mathop \approx \limits_{W} E_{9} \mathop \approx \limits_{W} E_{10} \mathop \approx \limits_{W} E_{11} \mathop \approx \limits_{W} E_{14} \mathop \approx \limits_{W} E_{15} \mathop \approx \limits_{W} E_{16} \mathop \approx \limits_{W} E_{19} ,} \\ \end{array} $$

(3)

where $\mathop \succ \limits_{W/S} ,\;\,\mathop \approx \limits_{W/S}$ are marks of comparative advantage and adequacy of hazard errors in GSP, constructed using the classical Wald’s criterion.

Table 1. Matrix of solutions for construction of group systems of advantages of air traffic controllers on dangers of characteristic errors (fragment).

The application of Wald's criterion can lead to the loss of a very successful solution (error of the first kind), when a significant error can get an inadequate rank for its danger. Therefore, in addition to it, other classical criteria of DM should be applied to the construction of ATC’s GSP on the dangers of characteristic errors.

Savage's criterion is usually seen as the development and refinement of Wald's criterion. This criterion is considered democratic for group decision-making because it takes into account the views of both the majority and the minority of experts. According to this criterion, such a strategy (GSP) is considered optimal, which provides the minimum general deviation from it of the ICP of respondents in the most unfavorable situation. This deviation is traditionally called risk, regret, fine, sadness.

The best solution (the most dangerous error) when applying the Savage criterion to the data in Table 1 is by the following formula:

$$ Z_{S} = \mathop {\min }\limits_{i} \;\mathop {\max }\limits_{j} \;a_{ij} = \mathop {\min }\limits_{i} \;\mathop {\max }\limits_{j} \;\left| {\mathop {\min }\limits_{i} \;r_{ij} - r_{ij} } \right|. $$

(4)

Consistent application of time (4) to the data of Table 1 led to the following GSP:

$$ \begin{array}{*{20}c} {E_{4} \mathop \succ \limits_{S} E_{20} \mathop \succ \limits_{S} E_{5} \mathop \approx \limits_{S} E_{18} \mathop \succ \limits_{S} E_{3} \mathop \succ \limits_{S} E_{21} \mathop \succ \limits_{S} E_{2} \mathop \succ \limits_{S} E_{12} \mathop \approx \limits_{S} E_{17} \mathop \succ \limits_{S} E_{13} \mathop \succ \limits_{S} E_{8} \mathop \succ \limits_{S} } \\ {\,\mathop \succ \limits_{S} E_{1} \mathop \succ \limits_{S} E_{7} \mathop \succ \limits_{S} E_{6} \mathop \approx \limits_{S} E_{9} \mathop \approx \limits_{S} E_{10} \mathop \approx \limits_{S} E_{11} \mathop \approx \limits_{S} E_{14} \mathop \approx \limits_{S} E_{15} \mathop \approx \limits_{S} E_{16} \mathop \approx \limits_{S} E_{19} ,} \\ \end{array} $$

(5)

where $\mathop \succ \limits_{W/S} ,\;\,\mathop \approx \limits_{W/S}$ are marks of comparative advantage and adequacy of hazard errors in GSP, constructed using the classical Savage’s criterion.

As we can see, in our specific case GSP (3) and (5), obtained using the Wald’s and Savage’s criterions, respectively, are identical.

The Bayes-Laplace criterion is unusually simple and comes down to finding the sum of the error ranks, their further averaging, and the ordering of the errors in ascending order of the calculated averages. Which corresponds to the application to the data of Table 1 of the following formula:

$$ Z_{BL} \; = \;\mathop {\min }\limits_{i} \;\overline{r}_{i} = \mathop {\min }\limits_{i} \;\left( {\frac{1}{\;n\;}\sum\limits_{j = 1}^{n} {r_{ij} } } \right). $$

(6)

where $\overline{r}_{i}$ is the rank of the i-th error, obtained by summing and averaging the opinions (ranks) of all m respondents-ATC’s.

Therefore, applying formula (6) to the data of Table 1, we obtain the following GSP:

$$ \begin{array}{*{20}c} {Er._{18} \mathop \succ \limits_{BL} Er._{20} \mathop \succ \limits_{BL} Er._{4} \mathop \succ \limits_{BL} Er._{21} \mathop \succ \limits_{BL} Er._{5} \mathop \succ \limits_{BL} Er._{3} \mathop \succ \limits_{BL} Er._{2} \mathop \succ \limits_{BL} Er._{8} \mathop \succ \limits_{BL} Er._{13} \mathop \succ \limits_{BL} Er._{16} \mathop \succ \limits_{BL} Er._{17} \mathop \succ \limits_{BL} } \\ {\mathop \succ \limits_{BL} Er._{12} \mathop \succ \limits_{BL} Er._{19} \mathop \succ \limits_{BL} Er._{6} \mathop \succ \limits_{BL} Er._{1} \mathop \succ \limits_{BL} Er._{7} \mathop \succ \limits_{BL} Er._{14} \mathop \succ \limits_{BL} Er._{11} \mathop \succ \limits_{BL} Er._{9} \mathop \succ \limits_{BL} Er._{10} \mathop \succ \limits_{BL} Er._{15} } \\ \end{array} $$

(7)

where $\mathop \succ \limits_{BL}$ are marks of the advantage of the danger of one error over another in the GSP, constructed using the Bayes-Laplace criterion.

Note that the Bayes-Laplace criterion led to a GSP of the form (7), which duplicates such a strategy of group decisions as summation and averaging of ranks, illustrated by the GSP (1). Therefore, the GSP obtained with its help is checked for consistency using the Kendall concordance coefficient. This is important, because when it comes to averaging ranks, it contributes to a risky result, when the found average value of a certain rank does not correspond to any opinion (Fig. 1).

The initial position of ATC as a person being DM, when applying the Bayes-Laplace criterion, is more optimistic than in the case of Wald's criterion, but assumes a higher level of awareness and long and frequent implementation. Therefore, this criterion is also called the criterion of insufficient justification.

Fig. 1.

Illustration of the dangers of simply averaging conflicting and opposing opinions about the dangers of E_i mistake.

The Hurwiz’s criterion is based on the desire to take an equilibrium position when choosing an indicator (rank) that best characterizes the risk of error. To do this, enter the coefficient of optimism $\alpha$ ($0 \le \alpha \le$) and the corresponding coefficient of pessimism $1 - \alpha$. The value of the coefficient is determined based on the initial (more optimistic or pessimistic) position of the expert. It is assumed that the danger of each error is best characterized by the weighted sum of the highest and lowest rank given to it in the ISP:

$$ Z_{HW} = \mathop {\min }\limits_{i} \;c_{i} = \mathop {\min }\limits_{i} \;\left[ {\alpha \cdot \mathop {\min }\limits_{j} \;r_{ij} + \left( {1 - \alpha } \right) \cdot \mathop {\max }\limits_{j} \;r_{ij} } \right]. $$

(8)

We assume that the coefficient of optimism is equal to α = 0.3, then, accordingly, the pessimism coefficient is equal to 1 – α = 0.7.

Applying the following values of the coefficients of optimism / pessimism and expression (8) to the data of Table 1, we obtain the following GSP:

$$ \begin{array}{*{20}c} {E_{4} \mathop \succ \limits_{HW} E_{20} \mathop \succ \limits_{HW} E_{18} \mathop \succ \limits_{HW} E_{5} \mathop \succ \limits_{HW} E_{3} \mathop \succ \limits_{HW} E_{21} \mathop \succ \limits_{HW} E_{13} \mathop \succ \limits_{HW} E_{2} \mathop \succ \limits_{HW} E_{1} \mathop \succ \limits_{HW} E_{12} \mathop \succ \limits_{HW} E_{8} \mathop \succ \limits_{HW} } \\ {\mathop \succ \limits_{HW} E_{6} \mathop \approx \limits_{HW} E_{16} \mathop \approx \limits_{HW} E_{19} \mathop \succ \limits_{HW} E_{7} \mathop \succ \limits_{HW} \,E_{11} \mathop \succ \limits_{HW} E_{9} \mathop \succ \limits_{HW} E_{15} \mathop \succ \limits_{HW} E_{14} \mathop \succ \limits_{HW} E_{10} \mathop \succ \limits_{HW} E_{17} } \\ \end{array} $$

(9)

where $\mathop \succ \limits_{HW} ,\;\,\mathop \approx \limits_{HW}$ are the corresponding marks of comparative advantage and adequacy of the risk of errors in the GSP, obtained using the Hurwitz’s criterion.

Note that the measuring properties of the ordering scale used to determine the ranks of error hazards impose certain restrictions on their mathematical processing. Therefore, when applying the Bayes-Laplace criterion, we used a simple sum of ranks. In addition, when applying the Hurwitz’s criterion, it was assumed that the scale is linear and quantitative.

4 Determination of the Risk Index of Indistinguishability of Error’s Danger

GSP (3), (5), (7), (9) were based on an additional factor of Spearman’s rank correlation. Its empirical values equal: $R_{S}^{BL - W/S}$ = 0.8922, $R_{S}^{BL - HW}$ = 0.9263, $R_{S}^{W/S - HW}$ = 0.9477, and is statistically plausible at an unusually high significance level for HF studies:

$$ \begin{array}{*{20}c} {t_{emp.}^{BL - W/S} = 8.611 > > t_{1\% ,\,k = 19} = 2.861;t_{emp.}^{BL - HW} = 10.716 > > t_{1\% ,\,k = 19} 2.861} \\ {t_{emp.}^{W/S - HW} = 12.943 > > t_{1\% ,\,k = 19} = 2.861.} \\ \end{array} $$

The above means that the coincidence of the ranks of the danger of errors in the GSP (3), (5), (7), (9) is a regularity, and not coincidence is an accident.

Based on the presence/absence of related ranks in the obtained GSPs (3), (5), (7), (9), it seems possible to calculate the following normalized indicators of the degree of fragmentation/indistinguishability of the danger of errors in them:

$$ R^{ * } = \frac{T}{{T_{\max } }} = \frac{{\sum\limits_{\gamma = 1}^{n} {\left( {t_{\gamma }^{3} - t_{\gamma } } \right)} }}{{n^{3} - n}}, $$

(10)

where $T$ is the indicator of the presence of related ranks in the GSP, which is determined from the formula for calculating the Kendall concordance coefficient. It makes sense of the correction factor, which is calculated in all $k$ “cases” of indistinguishability of ordered objects-errors; $t_{\gamma }$ is the number of indistinguishable errors of one “case”; $n = 21$ is the number of errors ranked; $T_{\max }$ is the indicator of maximum error indistinguishability, when all ordered errors are conditionally considered to be the same in terms of danger:

$$ E_{1} = E_{2} = ... = E_{n} \; \Leftrightarrow \;r_{{E_{1} }} = r_{{E_{2} }} = ...r_{{E_{21} }} \; \Leftrightarrow \;T_{\max } = n^{3} - n = 21^{3} - 21 = 9240. $$

(11)

If condition (11) is really fulfilled, then the error indistinguishability index in the GSP is the maximum and is equal to $R^{*} = R_{max}^{*} = 1$. If, on the contrary, all errors are strictly ordered, that is, there are no associated (middle) ranks in the GSP, this indicator is minimal, this figure is minimal $R^{*} = R_{\min }^{*} = 0$.

Applying formulas (10), (11) to the data of Table. 1 and GSP (3), (5), (7), (9), we obtain $R_{BL}^{*} = 0$, which is quite natural, since in GSP (7), which was obtained using the Bayes-Laplace criterion, there are no related ranks.

The studied indicator reaches the maximum (among the obtained) values $R_{W/S}^{*}$ = 5.58⋅10^–2 for GSP (3), (5), constructed using Wald/Savage criteria. Which is 29 times more than for GSP (9), built using the Hurwitz criterion: $R_{HW}^{*}$ = 0.19⋅10^–2.

Note that although the absolute values of the established empirical indicators $R^{*}$ are small, the results still give an idea of the comparative effectiveness of the applied classical criteria of DM for risk assessment – the uncertainty of the indistinguishability of the dangers of errors in them. To assess the degree of differentiation of the dangers of errors by the expert group as a whole, expression (10) is converted into the following:

$$ R_{g}^{*} = \frac{1}{m}\sum\limits_{{j = 1}}^{m} {R_{j}^{*} } = \frac{1}{m}\sum\limits_{{j = 1}}^{m} {\frac{{\sum\limits_{{\gamma = 1}}^{n} {\left( {t_{{\gamma j}}^{3} - t_{{\gamma j}} } \right)} }}{{n^{3} - n}}} = \frac{1}{{m\left( {n^{3} - n} \right)}}\sum\limits_{{j = 1}}^{m} {\sum\limits_{{\gamma = 1}}^{n} {\left( {t_{{\gamma j}}^{3} - t_{{\gamma j}} } \right)} } , $$

(12)

where $R_{j}^{*}$ is the indicator of the risk of indistinguishability of the dangers of errors in the ISP of the j-th expert-ATC; $t_{\gamma j}$ is the number of indistinguishable errors of one “case” in the ISP j-th ATCs.

Using formula (12) and the data in Table. 1, we establish that $R_{g}^{*}$ = 0.52⋅10^–2. As we can see, this indicator is almost identical to the result obtained for GSP (9), constructed using the Hurwitz criterion.

5 Conclusions and Prospects for Further Research

Based on the new scientific results obtained and presented in this publication. It is necessary to state the fact of a real solution to the problem of correct application of the spectrum of classical decision making (DM) criteria (Wald, Savage, Bayes-Laplace, Hurwitz) for construction of group system of preferences (GSP) of Ukrainian ATC’s on characteristic errors, which they make in their professional activities. Some partial results include the following:

• 1. From the comparison of the obtained GSP follows the adequacy of the rankings the results of the application of the Wald and Savage criterion, as well as the Bayes-Laplace criterion and such a strategy of group decisions as summation and averaging of ranks.

• 2. All the obtained group systems of preferences (GSP) coincide, confirming the unusually high in absolute value positive values of Spearman's rank correlation coefficients, statistically significant at an unusually high level of significance $\alpha = 1\%$ for human factor (HF) studies.

• 3. The normalized risk factor is introduced – the uncertainty of the indistinguishability of alternatives-errors, based on one of the components of the formula for determining the Kendall concordance coefficient. The minimum risk of indistinguishability is observed in the GSP obtained using the Bayes-Laplace criterion ($R_{BL}^{*}$ = 0), the maximum – under the conditions of application of the Wald/Savage criterion ($R_{W/S}^{*}$ = 5.58⋅10^–2). Some intermediate place is occupied by the results of the application of the Hurwitz test ($R_{HW}^{*}$ = 0.19⋅10^–2). At the same time, the error indistinguishability index for the expert group as a whole reaches a value and is close to the indicator calculated for the GSP, determined using the Hurwitz criterion.

• 4. The given methodology of application of classical criteria of DM is universal and can be applied to construction of GSP for researches in any field of human activity.

• 5. Based on the above, it should be noted the fact of expanding the methodology of expert procedures in the study of the human factor (HF). Further research should be conducted in the following areas (without ranking):

• construction of the Kemeny’s median as an optimization ATC’s GSP on the spectrum of characteristic errors;

• carrying out of the comparative analysis of efficiency of methods of construction of ATC’s GSP on dangers of a spectrum of characteristic errors;

• clarification of the possible influence of cross-cultural factors on the attitude of ATC’s to the dangers of typical errors and etc.