Keywords

1 Introduction

Computer simulation-based DoEs are used in a wide range of applications to learn about the effect of input variables x on a response of interest y. Fang et al. (2006) said, as simulation programs are usually deterministic so the output of a computer experiment is not subject to random variations, which makes the design of computer experiments different from that of physical experiments. Many simulation models involve several hundred factors or even more. It is desirable to avoid replicates when projecting the design onto a subset of factors. This is because a few out of the numerous factors in the system usually dominate the performance of the product. Thus, a good model can be fitted using only these few important factors. Therefore, when we project the design on to these factors, replication is not required. As it is recognized by several authors, the choice of the design points for computer experiments should at least fulfill two requirements namely non-collapsing and space-filling (Johnson et al. 1990; Morris and Mitchell 1995). In addition, in order to find out the individual effect of factors on response surface, researchers hunt DoEs with orthogonal property among the factors. LHD inherently preserves non-collapsing property, i.e. factors are evenly spread whenever projected on any coordinate. McKay et al. (1979) first introduced LHD. Later, authors (Morris and Mitchel 1995; Grosso et al. 2009) defined LHD which are a bit different. In a LHD, there are N points (design points) and each point has k distinct coordinates (parameters/factors). The points are placed in such a way that they are uniformly distributed when projected on every single coordinate (factor) axis. In a LHD, the range of each parameter/factor is normalized to the interval [1, N] (actually authors Grosso et al. 2009) considered the interval [0, N-1] but for the symmetry of other designs considered, here we have considered the range [1, N]). Mathematically, let us consider a set of N points in a uniform k-dimensional grid {1, 2, …, N}k:

$$X = \left( \begin{gathered} \user2{X}_{1} \hfill \\ \vdots \hfill \\ \user2{X}_{\user2{N}} \hfill \\ \end{gathered} \right) = \left( {\begin{array}{*{20}c} {x_{{11}} } & \ldots & {x_{{1k}} } \\ \vdots & \ldots & \vdots \\ {x_{{N1}} } & \ldots & {x_{{Nk}} } \\ \end{array} } \right)\forall x_{{ij}} = \left\{ {1,2, \ldots ,N} \right\}.$$

Then X is a LHD iff \(x_{{pj}} \, \ne \,x_{{qj}} \,\forall \,j,p,q\, \in \,\left\{ {1,2, \ldots ,N} \right\}\,\exists \,p \ne q\), i.e. each column has no duplicate entries. Note that the number of possible LHDs is huge: there are (N!)k possible LHDs.

It is known that randomly generated LHD has poor space-filling property. Moreover, it bears poor orthogonal property. Therefore, researchers search for optimal LHDs in the sense of space-filling, orthogonal, uniform etc., Jin et al. (2005), and Johnson et al. (1990) proposed Maximin (maximize the minimum inter-site pair-wise distance) experimental design but it was not LHD. They measured the space-filling on the basis of minimum inter-site pair-wise distance “D1” among all pair-wise distances of design points. That is if “D1” value of any LHD is relatively smaller then we say that the LHD is not good enough in the sense of space-filling. Morris and Mitchell (1995) proposed \({\mathrm{\Phi }}_{p}\) optimal criterion and later Grosso et al. (2009) modified it for easy computation, which is given below:

$$\Phi _{p} \left( X \right) = \sum\limits_{{i = 1}}^{N} {\sum\limits_{{j = i + 1}}^{N} {\left[ {\frac{1}{{d_{{ij}}^{p} }}} \right]} } ^{{\frac{1}{p}}}$$
(1)

where dij  =  d(xi, xj) be the distance between points xi and xj and p is a positive integer.

Audze and Eglais (1977) proposed a new optimal criterion that is similar to the potential energy function (Potential (U)) and which is defined as below:

$$Min\,\,U = \min \sum\limits_{{i = 1}}^{N} {\sum\limits_{{j = i + 1}}^{N} {\frac{1}{{d_{{ij}}^{2} }}} } .$$
(2)

It is known that multicollinearity (orthogonal property) measures the linear dependency among the factors of the design points. Multicollinearity may be measured by the partial pair-wise correlations among the factors. Several ways are available in the literature to measure the pair-wise correlations. Here, we consider the measure of average pair-wise correlations and maximum correlation, respectively, as below:

$${\rho }^{2}=\frac{\sum _{i=2}^{k}\sum _{j=1}^{i-1}{\rho }_{ij}^{2}}{k(k-1)/2}$$
(3)

and

$$\rho _{{\max }} = \mathop {\max }\limits_{{1 \le i,j \le k}} \rho _{{ij}}$$
(4)

Here ρij denotes the simple product–moment correlation between the factors i and j; k denotes the number of factors in the design considered. Butler (2001) found optimal LHD on the basis of this criterion. On the other hand, Joseph and Hung (2008) proposed the combination of the above two criteria. Moreover, Fang et al. (2000) defined a uniform design that allocates experimental points uniformly spread over the domain. It is noted that uniform design does not require orthogonal (multicollinearity) property. It considers projective uniformity over all sub-dimensions. But, in Fang et al. (2000), authors classified uniform design as space-filling design. A good literature review is available in Viana et al. (2014). It is worthwhile to mention here that Jamali et al. (2019) recently established some relations and bonds between Euclidian and Manhattan distance measures regarding LHD.

2 Experimental Result and Discussion

For the experimental study, we have considered a typical LHD of four factors (k = 4) with nine (N = 9) design points, which is optimized using several optimal criteria. The optimized LHDs are shown in Table 1. In Table 1, MLH-SA denotes Maximin LHD regarding Manhattan distance measure (L1) obtained by Simulated Annealing (SA) approach (Husslage et al. 2011). OMLH-MSA denotes Orthogonal Maximin LHD regarding L1 distance measure obtained by Modified Simulated Annealing (MSA) (Morris and Mitchell 1995), OLH-Y denotes Orthogonal LHD regarding L1 distance measure obtained by Ye (1998), ULH-F denotes uniform LHD regarding L1 distance measure obtained by Fang et al. (2000) and MLH-ILS indicates the Maximin LHD regarding Euclidean distance measure (L2) obtained by Iterated Local Search (ILS) approach (Grosso et al. 2009).

Table 1 Some optimal LHDs for (k,N) = (4,9)
Full size table

At first, we have extracted all important statistical characteristics of each optimal design given in Table 1. Then all the findings are shown in Table 2. The symbols SD, CoV, β1, β2, Min Distance, Max Distance denote Standard Deviation, Coefficient of Variation (%), Moment Coefficient of Skewness, Moment Coefficient of Kurtosis (\({\alpha }_{1}+3\)), minimum pair-wise inter-site distance among all design points of the LHD and maximum pair-wise inter-site distance among all design points of the LHD respectively. Besides, D1L1 and D1L2 denote the minimum pair-wise inter-site distance of design points measured in L1 (Manhattan) distance measure and in L2 (Euclidean) distance measure respectively. It is mentioned earlier that space-filling is defined on only D1 value or \(\mathrm{\Phi }\) value. Similarly, orthogonal LHD is defined by multi-collinearity and so on. But the insight views of several optimized LHDs are still unknown. Here, we have analyzed the optimal LHDs statistically and on distance measures. In addition, we have compared optimized LHDs in those issues.

Table 2 Statistical analysis among several optimal LHDs for (k, N) = (4, 9)
Full size table

We have also detected all pair-wise inter-site Euclidean distances (actually square of Euclidean distances) “Di,j2” values for all pair of design points for each optimal LHDs. Now for each optimal LHD, the “Di,j2” are plotted corresponding to each pair of design points (I = {1,2,..., 36}), which are shown in Fig. 1. It is noted that since we have taken 9 design points, we have obtained [(N (N – 1)/2] = 36 number of pair-wise distances. It is worthwhile to mention here that the sum of square of all pair-wise (Euclidian) distances,\({\left(\mathrm{d}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)\right)}^{2}\), of a fixed LHD is identical (=2160) and so average of the square distances of all LHDs (for k = 4, N = 9) are also identical. It is noted that there are (9!)4 such LHDs and for each such LHD the value of \(\sum {\left(\mathrm{d}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)\right)}^{2}\) is equal to 2160.

Fig. 1
figure 1

Graphical representation of square of Euclidean distances corresponding to the pair of design points for each optimal LHDs where (k, N) = (4, 9). Source Created by the authors

Full size image

Now, we will discuss the comparison between two Maximin LHDs namely MLH-SA and MLH-ILS in which both are optimized by using same single optimal criterion called Maximin optimal criterion. But distance measures namely L1 and L2 are used in MLH-SA and MLH-ILS, respectively, which are different. In addition, different optimal heuristic algorithms namely SA and ILS approaches are applied to optimize the MLS-SA and MLH-ILS designs, respectively. It is observed in both MLH-SA and MLH-ILS that the values of SD, CoV, \({\beta }_{1,}\) \({\beta }_{2},\) \(\rho\), \({\rho }_{Max}\), \({\mathrm{\Phi }}_{p}^{L2}\) and Potential (U) are not significantly different. It is also observed that the difference of D1L1 of the two DoEs is not significant, though MLH-SA searched optimal D1L1 value whereas MLH-ILS explored for D1L2 value rather than D1L1.

Now we will talk about the comparison between Maximin LHD and Uniform LHD mainly MLH-ILS and ULH-F. It is shown in Table 2 that uniform optimality criterion-based LHD is flatter and un-skewed compared to all other LHDs considered here. Moreover, the coefficient of correlation of ULH-F is relatively smaller than that of MLH-ILS. It is also observed that SD, CoV, Max Distance and Range values are not significantly different from the two optimal LHDs namely MLH-ILS and ULH-F. It is also clear from Table 2 that the D1L1 values of both DoEs are identical. On the other hand, for \({\mathrm{\Phi }}_{p}^{L2}\), Potential (U) and D1L2 values, MLH-ILS design is significantly better than those of ULH-F design. It is worthwhile to mention here that the D1L2 value of ULH-F is worst compared to other optimal designs considered here.

We will also discuss orthogonal-based DoEs mainly of OLH- Y. In addition, comparison will be drawn with Maximin-based LHD mainly MLH-ILS. Since the initial criterion of OLH-Y is \(\rho =0\), so obviously the factors of the DoE are all uncorrelated. It is observed in Table 2 that the SD, CoV, \({\beta }_{1}\) and D1L1 values of OLH-Y are not significantly different with that of MLH-ILS. But it is noticed that for \({\beta }_{2}\), Max Distance, Range, \({\mathrm{\Phi }}_{p}^{L2}\), Potential (U) and D1L2 values, OLH-Y is significantly worse compared with the values of MLH-ILS. Moreover, though in OLH-Y, L1 distance measure is used whereas, in MLH-ILS, L2 distance measure is used but both D1L1 values are identical. On the other hand, D1L2 value of MLH-ILS is significantly better compared with not only OLH-Y design but also all other designs considered here.

Now we will discuss the properties of OMLH-MSA in which multi-objective function namely Maximin and multi-collinearity was considered. It is observed that the SD, the CoV, the \({\beta }_{1}\) and Max Distance values of OMLH-MSA are the best compared with all other optimal designs. But these values are not significantly different from those values of MLH-ILS.

It is observed in Table 2 that the average correlation as well as maximum correlation of OMLH-MSA is negligible and smaller than those values of MLH-ILS. But it is also noticed in Table 2 that the average correlation as well as maximum correlation of MLH-ILS is also not significantly larger. In addition, it is also observed in Table 2 that D1L1 value of OMLH-MSA is not significantly smaller than that of other optimal LHDs. On the other hand, D1L2, \({\mathrm{\Phi }}_{p}^{L2}\) and potential (U) values of MLH-ILS are the best compared with all other optimal LHDs. Moreover, these values are significantly better compared with OMLH-MSA. It is also observed in Fig. 1 that according to space-filling criterion MLH-ILS is relatively looking more homogeneous over the design space compared with other designs.

Now we will discuss the two distance measures considered here namely Euclidean and Manhattan. It is noticed in Table 2 that except MLH-ILS all other designs are optimized by using Manhattan distance measure whereas MLH-ILS is optimized by considering Euclidean distance measure. It is observed in Table 2 that D1L1 value of anyone LHD compared with the remaining other LHDs is not significantly better. On the other hand, the D1L2 value of MLH-ILS is the best and significantly larger than any other one.

3 Conclusion

When LHD is considered as an experimental design, the design inherently preserves good non-collapsing property. On the other hand, when Maximin optimal criterion is considered, then the DoE should have good space-filling property. Again, when we use multicollinearity as an optimal criterion, then the factors of the DoE are uncorrelated or approximately uncorrelated. Similarly, whenever the uniform optimal design is considered then design becomes flatter. Besides optimal criteria, another question is frequently raised; which distance measure is relatively better in case of an optimal design? In this elementary study, the main objective is to find out the schematic view of several well-known optimal designs from the statistical point of view. A rigorous statistical analysis has been carried out over some well-known optimal LHDs, which are optimized with different optimal criteria. From this elementary analysis, it seems to us that, if we prefer a DoE containing all three properties (good) then multi-objective-based LHD namely OMLH-MSA is relatively better. But if one considers only space-filling LHD, then MLH-ILS is the best in which Euclidean distance measure is considered. Similarly, if one needs the design with uncorrelated factors along with non-collapsing property then it is obvious that OLH-Y is the best. In addition, it seems to us that L2 is relatively better to find out Maximin LHD. Though multi-objective-based optimal LHD is relatively better, it is obviously time-consuming. Moreover, in that optimal DoE, none of the individual optimal criterion (of multi-objective function) is found good enough. It is worthwhile to mention here that ILS is preferable to find Maximin LHD with cheaper computational time. Therefore, according to us, one may first find out an optimal LHD by considering Maximin optimal criterion with ILS approach to detect the important factors on response surface by using L2 distance measure. Then orthogonal optimal criteria may be applied on the reduced DoE (eliminated unimportant factors) to get an uncorrelated-based DoE for further experiments. But it is worthwhile to mention here that, in this experimental research, only some optimal LHDs of (k, N) = (4, 9) are considered. At the same time, we would like to note here that in the existing literature only four optimal LHDs of (k, N) = (4, 9) viz MLH-SA, OMLH-MSA, OLH-Y, ULH-F, MLH-ILS are available, which means, some other instances that are optimized with all these five approaches are not yet available in the literature. So, further experiments (with different instances) should be carried out to draw a more concrete conclusion.