Optimum Addition of Information to Computer Experiments in View of Uniformity and Orthogonality

Abstract

Computer experiments have become ubiquitous across the engineering, physical and chemical sciences. Computer experiments are constructed to emulate the behavior of a physical system. Assume that we perform an experiment using a two-level uniform design. If, after obtaining data, we decide additional runs of the computer simulator are needed, how to add more runs after collecting our data? How to design the experiment to efficiently extract useful information from it? In this paper, we try to answer these questions by providing a new approach for constructing efficient uniform designs by adding new runs to an existing uniform design. The optimization criteria are the uniformity criteria measured by Lee, symmetric, wrap-around, centered and mixture discrepancy and the orthogonality criteria measured by the B-criterion and the O-criterion. We investigate the relationship between orthogonality and uniformity criteria in view of analytical expressions and lower bounds.

1 Introduction

Computer simulations based on mathematical models have been widely applied in industry and high-technology development. Computer experiments are used to make inferences about the behavior of physical systems because they are often easier, cheaper and faster than physical experiments to perform. For example, the Big Bang models are often used because physical experimentation is impossible. Unlike physical experiments with known models, computer experiments are often performed with little knowledge about the relation between the responses (outputs of the simulator) and the inputs before experimentation.

The mission of constructing efficient experimental designs is a major challenge in scientific research. Most real-life projects embrace high-dimensional inputs with limited available resources. These limited available resources can be used more efficiently. A good experimental design should minimize the number of runs needed to acquire as much information as possible. Therefore, we need to find a way to experiment so that as much information as possible could be found using relatively few experimental points. The fundamental idea is that the design should spread the experimental points uniformly over the experimental domain so that no region is missed. Such designs are called space-filling designs.

The uniform design (UD) is a pioneer type of space-filling design that was suggested by Fang [11] and Wang and Fang [26]. The UD has been widely applied in manufacturing, system engineering, pharmaceutics and natural sciences. The UD is a type of robust design of experiments. This is essential to model projects in which the relation between the outputs and inputs is not well known. At the initial stage of an experiment, it is often the case that the experimenter does not have enough information about the relation between the outputs and the inputs. Therefore, it is important to use a design that is robust against the underlying model specifications. Since the UD spreads the design points evenly in the design space, it usually has robust performance under different models. Hickernell [20] and Yue and Hickernell [28] proved that the UD is optimal and robust for approximate linear regression methods. Moreover, Xie and Fang [27] proved that the UD is admissible and minimax in a certain sense in nonparametric regression models.

In order to measure the uniformity, various uniformity criteria (discrepancies) have been proposed including the Lee discrepancy $({\mathcal {LD}})$, symmetric $L_2$-discrepancy $({\mathcal {SD}})$, wrap-around $L_2$-discrepancy $({\mathcal {WD}}),$ centered $L_2$-discrepancy $({\mathcal {CD}})$ and mixture $L_2$-discrepancy $({\mathcal {MD}}).$ The reader can refer to Zhou et al. [31], Zhou et al. [30] and Hickernell [18, 19]. A uniform design minimizes the discrepancy value. The research of developing efficient methodologies for constructing UDs has been very active in the last decade. Using one of the discrepancies as the objective function, to find a UD is an NP hard optimization problem in the sense of computational complexity. Therefore, for reducing the computational complexity some structure for experimental points has to be considered. Instead of optimizing over a very large set of all possible n-run designs, one may find a very good design by considering a much smaller candidate set provided that it contains low-discrepancy designs. One of such sets is the set of balanced levels designs, or the so-called U-type designs, proposed by Fang and Hickernell [12].

A symmetric balanced design $\mathcal {B}(n; q^m)$ corresponds to an $n\times m$ matrix $X = (x_1, \ldots , x_m)$ such that each column $x_i$ takes values from a set of q integers, say ${0, 1, \ldots , q-1},$ equally often. Denote by $\mathbb {B}(n; q^m)$ the set of all $\mathcal {B}(n; q^m)$ designs. By mapping $f : l \rightarrow (2l+1)/(2q), l = 0, \ldots , q-1,$ the n runs are transformed into n points in $C^m = [0, 1]^m.$ On the other hand, throughout our paper we give another set of designs which are called symmetric nearly balanced designs $\mathcal {N}(n; q^m)$ corresponding to an $n\times m$ matrix $X = (x_1, \ldots , x_m)$ such that each column $x_i$ takes values from a set of q integers, say ${0, 1, \ldots , q-1},$ as equally often as possible. Denote by $\mathbb {N}(n; q^m)$ the set of all $\mathcal {N}(n; q^m)$ designs. It is to be noted that any design $\mathcal {N}\in \mathbb {N}(n; q^m)$ may be balanced or nearly balanced depending upon whether n is a multiple of q or not, respectively, i.e., when n is a multiple of q the design is balanced and when n is not a multiple of q,  we can find a design $\mathcal {B} \subset \mathcal {N}$ such that $\mathcal {B}\in \mathbb {B}(t; q^m),$ where t is the integral part of n / q (i.e., $\mathcal {B}$ is a balanced design with t runs).

It is an important issue to find good lower bounds for the discrepancy measure of uniformity, because lower bounds can be used as benchmarks in searching for UDs. A design whose discrepancy value achieves a sharp lower bound is a UD with respect to this discrepancy. In this regard, mention may be made to Fang and Mukerjee [15]. The work of Fang and Mukerjee [15], on regular fractions of two-level factorials, was a first attempt towards providing a lower bound for the ${\mathcal {CD}}$. Fang et al. [13] extended their results to nonregular two-level fractional factorials and gave lower bounds with respect to the ${\mathcal {CD}}$ and the ${\mathcal {WD}}$ (both regular and nonregular). Fang et al. [16] and Qin et al. [24] provided tighter lower bounds of the ${\mathcal {CD}}$ and ${\mathcal {WD}}$ for two-level and three-level factorials. Fang et al. [14] provided lower bounds for three- and four-level designs under the ${\mathcal {CD}}$. Later, many authors have been studying along this direction. Elsawah et al. [3, 4] obtained a new lower bound of the ${\mathcal {CD}}$ under four-level designs and the lower bound of the ${\mathcal {CD}}$ for mixed two- and three-level designs, respectively. Recently, Elsawah et al. [5] obtained new lower bounds of the ${\mathcal {MD}}$ based on two-, three- and four-level designs. Finally, Elsawah et al. [9] discussed the uniformity of mixed two- and three-level designs in view of the ${\mathcal {MD}}.$

2 Organizational Structure and Strategies Implementation

Suppose that a practitioner begins with an experiment using a two-level uniform balanced design $\mathcal {B} \in \mathbb {B}(n; 2^m)$. The assumptions the experimenter makes in selecting an initial design may be incorrect, and after seeing data the experimenter may realize the initial design $\mathcal {B}$ is inadequate and must be supplemented by additional r more runs. The design consisting of r runs is called as added design, denoted by $\mathcal {A} \in \mathbb {N}(r; 2^m).$ It is to be noted that the design $\mathcal {A}$ may be balanced or nearly balanced depending upon whether r is even or odd, respectively, i.e., the levels of each factor occur as equally often as possible. The full design obtained by augmenting the runs of the design $\mathcal {A}$ to those of the original uniform or nearly uniform design $\mathcal {B}$ is called as extended design, denoted by $\mathcal {E}$, that is, $\mathcal {E}=(\mathcal {B}'$  $\mathcal {A}')'.$ The extended design will also be balanced or nearly balanced depending upon whether $n+r$ (r) is even or odd, respectively. Denote by $\mathbb {E}(n+r; 2^m)$ the set of all extended designs $\mathcal {E}.$

An illustrative example Suppose that an experimenter begins the experimentation using a uniform design of 5 factors with 4 levels and 8 runs. According to his assumptions and the available resources he/she can choose a suitable design table $\mathcal {P}_1\in \mathbb {B}(8,4^5)$ from the uniform design site (http://sites.stat.psu.edu/~rli/DMCE/UniformDesign/). After seeing data, after the experiment is over or during the experimentation, the experimenter may realize the initial design $\mathcal {P}_1\in \mathbb {B}(8,4^5)$ is inadequate and must be supplemented by additional four more runs. From the uniform design site, the following design table $\mathcal {P}_2\in \mathbb {B}(12,4^5)$ is the uniform design for 12 runs. From Table 1, we can show that the 12 runs are completely different from 8 runs, i.e., the level combinations are completely different, where there are no any common runs. This means that there is no way to add the new 4 runs to his/her original 8 runs uniform design $\mathcal {P}_1$ so as to get the existing 12 runs uniform design $\mathcal {P}_2$. Therefore, a new approach for constructing uniform or nearly uniform designs by adding new runs to an existing uniform design is needed.

Table 1 Four-level uniform designs

The extended designs have been applied in numerical integration, microarray experiments and computer experiments, see Loeppky et al. [22], Durrieu and Briollais [1] and Tong [25]. In particular, Ji et al. [21] described a sequential procedure for the method of development of fingerprints based on a uniform design approach, in which the sequential uniform design is used to reach the global optimum for a separation.

Now comes to the mind the following intuitive question, how does the experimenter select the additional runs and augment the original design $\mathcal {B}$ so as to get an extended design $\mathcal {E}(n+r; 2^m)$ which is uniform or nearly uniform under the given measure of uniformity? Thus, our mission is to construct an extended uniform (nearly uniform) design $\mathcal {E}(n+r; 2^m)$ given that the original design $\mathcal {B}\in \mathbb {B}(n; 2^m)$ is a uniform or a nearly uniform design. This situation can be handled in the following three different possible strategies.

• 1. The first possible strategy: We may construct a uniform extended design $\mathcal {E}(n+r; 2^m)$ using the existing lower bounds (given below) depending upon whether $(n+r)$ is even or odd for evaluating the efficiency of the extended design.

  This strategy is not a feasible solution because the experimenter has already run the experiment with n design points and r additional runs have to be added with the existing runs.

• 2. The second possible strategy: We may add r runs to the existing uniform design $\mathcal {B}\in \mathbb {B}(n; 2^m)$ and use the existing bounds (given below) for evaluating the efficiency of the extended design $\mathcal {E}(n+r; 2^m)\in \mathbb {N}(n+r; 2^m).$

  This strategy is not a good solution because these lower bounds maybe difficult to attain because the subdesign with n runs is already uniform (nearly uniform) using these bounds. See Theorem 8, Remarks 6 and 7 and Examples 1 and 2 in Sect. 6.

• 3. The third possible strategy: We may construct a uniform (nearly uniform) design for the added r runs $\mathcal {A}\in \mathbb {N}(r; 2^m)$ using the existing lower bounds (given below) depending upon whether r is even or odd for evaluating the efficiency of the added design and augment the original design $\mathcal {B}\in \mathbb {B}(n; 2^m)$ so as to get the extended design $\mathcal {E}(n+r; 2^m).$

  This strategy is not an effective solution because it may not be the case that the union of any two uniform (nearly uniform) designs yields a uniform (nearly uniform) extended design. See Example 3 and Remark 7 in Sect. 6.

There is, therefore, a need to obtain a new strategy (improving strategy 2) for constructing optimal extended designs $\mathcal {E}(n+r; 2^m),$ obtained by adding r new runs to $\mathcal {B}\in \mathbb {B}(r; 2^m),$ given that $\mathcal {B}$ is a uniform (nearly uniform) design. The main objective of the present paper is to provide an answer to the question: How to choose the r runs optimally?

This paper considers the study of the optimality of the extended design generated by adding a few runs to an existing two-level uniform design. The criteria of the optimality of additional runs are the uniformity criteria measured by Lee, symmetric, wrap-around, centered and mixture discrepancy. For two-level uniform factorials as the original designs, we investigate new analytical expressions of the above discrepancies for extended designs in view of Kronecker product, Schur-convex function and row (Hamming) distance. We derive results connecting uniformity and orthogonality and show that these criteria agree quite well, which provide further justifiable interpretation for two measures of orthogonality by the consideration of uniformity. This analysis provides new lower bounds for these discrepancies for symmetric two-level extended designs, which can be used as benchmarks for constructing optimal extended designs. Using the new formulations of the uniformity criteria and the new lower bounds as the benchmark, we can implement a new version of the fast local search heuristic threshold accepting ([16], see also [14]). By this search heuristic, we can obtain two-level extended designs with low discrepancy. Illustrative examples are provided, where numerical studies lend further support to our theoretical results. Our results provide a theoretical justification for the optimality of additional runs to uniform designs in terms of uniformity criteria.

The remainder of this paper is organized as follows. We discuss in depth the uniformity criteria (UC) for two-level balanced designs and their lower bounds with a comparison study in Sect. 3. In Sect. 4, we give a construction procedure of the extended design and obtain new lower bounds to all the above uniformity criteria for two-level extended designs. Connections between uniformity and orthogonality are made explicit in Sect. 5. The construction of optimal (uniform) extended designs has been illustrated through examples in Sect. 6 with discussions about the new strategy and the other possible strategies mentioned above, where numerical studies lend further support to our theoretical results. We close with the conclusions and possible extensions in Sect. 7.

3 UC and Their Lower Bounds with a Comparison Study

For any two-level balanced design $\mathcal {B}\in \mathbb {B} (n; 2^m),$ its ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ values, denoted, respectively, by ${\mathcal {LD}}(\mathcal {B}),$${\mathcal {SD}}(\mathcal {B}),$${\mathcal {WD}}(\mathcal {B}),$${\mathcal {CD}}(\mathcal {B})$ and ${\mathcal {MD}}(\mathcal {B}),$ can be expressed in the following forms, respectively

$$\begin{aligned}&[{\mathcal {LD}}(\mathcal {B})]^2=-\left( \frac{3}{4}\right) ^{m}+\frac{1}{n^{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}\prod _{k=1}^{m}\left[ 1-\alpha _{ij}^k\right] ,\\&[{\mathcal {SD}}(\mathcal {B})]^2 = \left( \frac{4}{3}\right) ^m-2 \left( \frac{11}{8}\right) ^m+\frac{2^m}{n^2}\sum _{i=1}^{n}\sum _{j=1}^{n}\prod _{k=1}^{m}(1-|u_{ik}-u_{jk}|),\\&[{\mathcal {WD}}(\mathcal {B})]^2 = -\left( \frac{4}{3}\right) ^m+\frac{1}{n^2}\sum _{i=1}^{n}\sum _{j=1}^{n}\prod _{k=1}^{m}\left[ \frac{3}{2}-|u_{ik}-u_{jk}|(1-|u_{ik}-u_{jk}|)\right] ,\\ {}[{\mathcal {CD}}(\mathcal {B})]^2= & {} \left( \frac{13}{12}\right) ^{m}-\frac{2}{n}\sum _{i=1}^{n}\prod _{k=1}^{m}\left[ 1+\frac{1}{2}\left| u_{ik}-\frac{1}{2}\right| -\frac{1}{2}\left| u_{ik}-\frac{1}{2}\right| ^{2}\right] \\&+\frac{1}{n^{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}\prod _{k=1}^{m}\left[ 1+\frac{1}{2}\left| u_{ik}-\frac{1}{2}\right| +\frac{1}{2}\left| u_{jk}-\frac{1}{2}\right| -\frac{1}{2}\left| u_{ik}-u_{jk}\right| \right] \end{aligned}$$

and

$$\begin{aligned}{}[{\mathcal {MD}}(\mathcal {B})]^2= & {} \left( \frac{19}{12}\right) ^{m}-\frac{2}{n}\sum _{i=1}^{n}\prod _{k=1}^{m}\left[ \frac{5}{3}-\frac{1}{4}\left| u_{ik}-\frac{1}{2}\right| -\frac{1}{4}\left| u_{ik}-\frac{1}{2}\right| ^{2}\right] \\&+\left. \frac{1}{n^{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}\prod _{k=1}^{m}\left[ \frac{15}{8}-\frac{1}{4}\left| u_{ik}-\frac{1}{2}\right| -\frac{1}{4}\left| u_{jk}-\frac{1}{2}\right| -\frac{3}{4}\right| u_{ik}\right. \\&\left. \quad -u_{jk}\left| +\frac{1}{2}\left| u_{ik}-u_{jk}\right| ^2\right] ,\right. \end{aligned}$$

where

$$\begin{aligned} \alpha _{ij}^k=\min \left\{ \frac{|x_{ik}-x_{jk}|}{2},1-\frac{|x_{ik}-x_{jk}|}{2}\right\} ,~u_{jk} \;=\; \frac{2x_{jk} + 1}{4}~\text{ and }~ x_{jk}\in \left\{ 0,1\right\} . \end{aligned}$$

The reader can refer to Zhou et al. [31], Zhou et al. [30] and Hickernell [18, 19].

For any two-level balanced design $\mathcal {B}\in \mathbb {B}(n;s^m),$ let

$$\begin{aligned}c= \left\{ \begin{array}{ll} -\left( \frac{3}{4}\right) ^{m}&{}\text{ for }~ {\mathcal {LD}}; \\ -\left( \frac{4}{3}\right) ^{m} &{}\text{ for }~ {\mathcal {WD}};\\ \left( \frac{4}{3}\right) ^{m} - 2 \left( \frac{11}{8}\right) ^{m}&{} \text{ for } ~ {\mathcal {SD}};\\ \left( \frac{13}{12}\right) ^{m}-2\left( \frac{35}{32}\right) ^{m}&{} \text{ for } ~ {\mathcal {CD}}; \\ \left( \frac{19}{12}\right) ^{m}-2\left( \frac{305}{192}\right) ^{m}&{} \text{ for } ~ {\mathcal {MD}}; \\ \end{array} \right. ~~ a= \left\{ \begin{array}{ll} \frac{1}{2}&{}\text{ for } ~ {\mathcal {LD}}; \\ \frac{5}{4}&{} \text{ for }~ {\mathcal {WD}};\\ \frac{3}{2}&{}\text{ for }~ {\mathcal {MD}};\\ 1 &{}\text{ for }~ {\mathcal {SD}}~ \text{ and }~ {\mathcal {CD}};\\ \end{array} \right. ~ b= \left\{ \begin{array}{ll} 2&{}\text{ for }~ {\mathcal {LD}} ~\text{ and }~ {\mathcal {SD}};\\ \frac{6}{5} &{}\text{ for }~ {\mathcal {WD}};\\ \frac{5}{4} &{}\text{ for }~ {\mathcal {CD}};\\ \frac{7}{6}&{}\text{ for }~ {\mathcal {MD}};\\ \end{array} \right. \end{aligned}$$

and $\Xi$ be a $2^m$-vector with components $n(i_1,\ldots ,i_m)$ arranged lexicographically, where $n(i_1,\ldots ,i_m)$ is the number of runs at the level combination $(i_1,\ldots ,i_m)$ in the design $\mathcal {B}.$ The vector $\Xi$ is called the frequency vector of the design $\mathcal {B}$, which means that the design $\mathcal {B}$ can be uniquely determined by the vector $\Xi$. By defining

$$\begin{aligned} \Omega =\mathfrak {R}\otimes \cdots \otimes \mathfrak {R}=\otimes _{i=1}^m\mathfrak {R},~\mathfrak {R}=\left( \begin{array}{cc} ab &{} a\\ a &{} ab \end{array}\right) ~\text{ and }~ \otimes ~ \text{ to } \text{ denote } \text{ the } \text{ Kronecker } \text{ product }, \end{aligned}$$

through application of the Kronecker calculus for factorial arrangements and an identification with a hypothetical full factorial as in Fang and Mukerjee [15] and Fang et al. [13], the above various discrepancy values on two-level balanced designs can be expressed in one equation in terms of $\Xi$, denoted by $[{\mathcal {VD}}(\mathcal {B})]^2,$ with lower bounds, denoted by $\mathcal {LBV}_{1\mathcal {B}},$ as in the following theorem.

Theorem 1

For any symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(n; 2^m)$, let $f_j$ be the integral part of $\frac{n}{2^j}$ and $z_j=n-2^jf_j.$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {B})]^2=c+\frac{a^m}{n^{2}}\Xi '\Omega \Xi \ge {\mathcal {LBV}}_{1\mathcal {B}}, \end{aligned}$$

(3.1)

where

$$\begin{aligned} \mathcal {LBV}_{1\mathcal {B}}=c+\frac{a^m}{n^{2}}\sum _{j=0}^{m}\left( \begin{array} {c} m \\ j \end{array} \right) \left( b-1\right) ^j \left[ nf_j+z_j(f_j+1)\right] . \end{aligned}$$

Recently, Zhang et al. [29] proposed the so-called majorization framework in which many criteria can be expressed as a Schur-convex function. The following theorem shows that the above various discrepancy values on two-level balanced designs can join this framework. Following the definition of the Schur-exponential criterion in Zhang et al. [29], the above various discrepancy values for two-level balanced designs can be expressed by a Schur-convex function, denoted by $[{\mathcal {VD}}(\mathcal {B})]^2,$ and then other lower bounds for these discrepancies, denoted by $\mathcal {LBV}_{2\mathcal {B}},$ are given by the following theorem.

Theorem 2

For any symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(n; 2^m)$, let $\theta =\frac{m(n-2)}{2(n-1)},~\Psi _E(\mathcal {B},b)$ be the Schur-exponential function and $\zeta$ and $\lambda$ are the integral part and fractional part of $\theta$, respectively. Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {B})]^2=c+\frac{a^mb^m}{n}+\frac{2a^m}{n^{2}}\Psi _E(\mathcal {B},b)\ge \mathcal {LBV}_{2\mathcal {B}}, \end{aligned}$$

(3.2)

where

$$\begin{aligned} \mathcal {LBV}_{2\mathcal {B}}=c+\frac{a^mb^m}{n}+\frac{(n-1)(1+\lambda ({b-1}))}{n}a^mb^\zeta . \end{aligned}$$

Finally, following Elsawah and Qin [6, 7] by defining $\zeta _{ij}$ be the number of places where the entries of the ith row and the jth row of the design $\mathcal {B}$ coincide, it is obvious that $\zeta _{ii}=m$ for $1\le i \le n$ and $\sum _{i=1}^{n}\sum _{j\ne i}^{n}{\zeta _{ij}}=nm(\frac{n}{2}-1),$ the above various discrepancy values on two-level balanced designs can be expressed in one equation in terms of $\zeta _{ij}$, denoted by $[{\mathcal {VD}}(\mathcal {B})]^2,$ with other lower bounds, denoted by $\mathcal {LBV}_{3\mathcal {B}},$ as in the following theorem.

Theorem 3

For any symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(n; 2^m)$, let $\zeta$ as in Theorem 2, $\alpha +\beta =n(n-1)$ and $\alpha \zeta +\beta (\zeta +1)=nm(\frac{n}{2}-1).$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {B})]^2=c+\frac{a^m}{n^{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}b^{\zeta _{ij}}=c+\frac{a^mb^m}{n}+\frac{a^m}{n^{2}}\sum _{i=1}^{n}\sum _{j\ne i}^{n}b^{\zeta _{ij}}\ge \mathcal {LBV}_{3\mathcal {B}},\nonumber \\ \end{aligned}$$

(3.3)

where

$$\begin{aligned} \mathcal {LBV}_{3\mathcal {B}}=c+\frac{a^mb^m}{n}+\frac{a^mb^{\zeta }}{n^{2}}(\alpha +\beta b). \end{aligned}$$

Figures 1, 2, 3, 4 and 5 give the comparison study between the lower bounds of the uniformity criteria measured by ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ given in Theorems 1, 2 and 3. In view of Figs. 1, 2, 3, 4 and 5, we need to provide the improved lower bounds of the various discrepancies as follows.

Corollary 1

For any symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(n;2^m),$ we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {B})]^2\ge \mathcal {LBV}_{\mathcal {B}}=\max \left\{ \mathcal {LBV}_{1\mathcal {B}},\mathcal {LBV}_{2\mathcal {B}},\mathcal {LBV}_{3\mathcal {B}}\right\} . \end{aligned}$$

(3.4)

Fig. 1

A comparison study between the lower bounds for $\mathcal {LD}$

Fig. 2

A comparison study between the lower bounds for $\mathcal {WD}$

Fig. 3

A comparison study between the lower bounds for $\mathcal {SD}$

Fig. 4

A comparison study between the lower bounds for $\mathcal {CD}$

Fig. 5

A comparison study between the lower bounds for $\mathcal {MD}$

4 Connections Between Uniformity and Orthogonality

Orthogonal designs have been used in various fields and form a major class of factorial designs. A design $d \in D(n; q^m)$ is called an orthogonal design, if all the level combinations for any two factors appear equally often. The orthogonal design is a special case of orthogonal arrays. An $n\times m$ matrix d with s symbols is said to be an orthogonal array of strength t, denoted by OA(n, m, q, t), if, for every $n\times t$ submatrix of d, the $q^t$ level combinations occur with the same frequency $n/q^t.$ Note that there exists an important projection property of an orthogonal array: When projected onto any t factors, it yields $n/q^t$ copies of a $q^t$ complete factorial. For a thorough discussion of orthogonal arrays, we may refer to Hedayat et al. [17] and Dey and Mukerjee [2].

Recently, some new criteria, such as the B-criterion [16] and the O-criterion [13], have been utilized to evaluate the orthogonality of designs. These criteria can be viewed as extensions of the concept of strength in an orthogonal array. We now briefly describe these two criteria as follows: For every t columns of $d \in D(n; q^m),~(d_{s_1}, d_{s_2}, \ldots , d_{s_t}),$ say, define

$$\begin{aligned} B_{{s_1},{s_2}, \ldots ,{s_t}}(d)=\sum _{\epsilon _1,\ldots ,\epsilon _t}\left( N_{\epsilon _1,\ldots ,\epsilon _t}^{({{s_1},{s_2}, \ldots ,{s_t}})}-\frac{n}{q^t}\right) ^2, \end{aligned}$$

where $N_{\epsilon _1,\ldots ,\epsilon _t}^{({{s_1},{s_2}, \ldots ,{s_t}})}$ is the number of runs in which $(d_{s_1}, d_{s_2}, \ldots , d_{s_t})$ takes the level combination $({\epsilon _1,\ldots ,\epsilon _t})$, and the summation is taken over all $q^t$ level combinations. If $B_{{s_1},{s_2}, \ldots ,{s_t}}(d)=0$, the subdesign formed by columns $(d_{s_1}, d_{s_2}, \ldots , d_{s_t})$ is an orthogonal array of strength t. Furthermore, define

$$\begin{aligned} B_t(d)=\frac{\sum _{1\le {{s_1}<{s_2}< \cdots <{s_t}}\le m} B_{{s_1},{s_2},\ldots ,{s_t}}(d)}{\left( \begin{array} {c} m \\ t \end{array} \right) }~\text{ for }~1\le t\le m. \end{aligned}$$

It is evident that $B_m(d) = 0$ if and only if an orthogonal array $d \in D(n; q^m)$ with strength t. Consequently, $B_m(d)$ measures the closeness to orthogonality of strength t of d. The vector $(B_1(d),B_2(d),\ldots , B_m(d))$ is a balance pattern introduced by Fang et al. [16], which measures the orthogonality of projections of d. In order to ensure proximity to orthogonal array of successively higher strengths, one should choose d so as to minimize $B_1(d),\ldots , B_m(d)$ sequentially. We call this criterion the B-criterion.

For comparing orthogonality of fractional factorial designs, Fang et al. [13] proposed another vector, denoted by $(O_1(d),\ldots , O_m(d))$ in this paper, to measure the orthogonality of d. The detailed definition of $O_i(d)$ can be found in Fang et al. [13]. The departure of d from being represented by an orthogonal array of strength t can be measured by $\sum _{i=1}^{t} O_i(d).$ The smaller $\sum _{i=1}^{t} O_i(d),$ the more design d behaves like an orthogonal array of strength t. Therefore, in order to ensure proximity to orthogonal array of successively higher strengths, one should choose d so as to minimize $O_1(d),\ldots , O_m(d)$ sequentially. We call this criterion the O-criterion. Recently, [23] showed that the O-criterion and the B-criterion are mutually equivalent as given in the following lemma.

Lemma 1

For any symmetric two-level design $d \in D(n; 2^m),$ the B-criterion coincides with the O-criterion, that is, sequentially minimizing $B_1(d),B_2(d),\ldots , B_m(d)$ is equivalent to sequentially minimizing $O_1(d),O_2(d),\ldots , O_m(d).$ In addition, the $B_t(d)$’s and $O_t(d)$’s are linearly related through the following formulae. For any $1\le t\le m$ and $1\le i \le m,$

$$\begin{aligned} B_t(d)=2^{m-t}O_i(d)\sum _{h=1}^t \left( \begin{array} {c} m-h \\ m-t \end{array} \right) /\left( \begin{array} {c} m \\ t \end{array} \right) \end{aligned}$$

and

$$\begin{aligned} O_i(d)=\left( \frac{1}{2}\right) ^m\left( \begin{array} {c} m \\ i \end{array} \right) \sum _{j=1}^i (-1)^{i-j}2^j\left( \begin{array} {c} i \\ j \end{array} \right) B_j(d). \end{aligned}$$

Now, let us consider the connection between uniformity measured by ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ and orthogonality measured by ${O_i(d)}$ or ${B_i(d)}.$ Following Fang et al. [13] and [23], the above various discrepancy values on two-level balanced designs can be expressed by ${O_i(d)}$ or ${B_i(d)}$, denoted by $[{\mathcal {VD}}({d})]^2,$ with other lower bounds based on the column balance of d, denoted by $\mathcal {LBV}^\mathcal {O},$ as in the following theorem.

Theorem 4

For any symmetric two-level design $d \in D(n; 2^m)$, let $\Pi _{n,i}$ is the residual of $n (mod~2^i).$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}({d})]^2= & {} c+a^m\left( \frac{1+b}{2}\right) ^m+\frac{a^m(b+1)^m}{n^2}\sum _{i=1}^m\left( \frac{b-1}{b+1}\right) ^iO_i(d)\\= & {} c+a^m\left( \frac{1+b}{2}\right) ^m+\frac{a^m}{n^2}\sum _{i=1}^m \left( \begin{array} {c} m \\ i \end{array} \right) ({b-1})^iB_i(d)\ge \mathcal {LBV}^\mathcal {O}, \end{aligned}$$

where

$$\begin{aligned} \mathcal {LBV}^\mathcal {O}=c+a^m\left( \frac{1+b}{2}\right) ^m+\frac{a^m}{n^2}\sum _{i=1}^m \left( \begin{array} {c} m \\ i \end{array} \right) \left( 1-\frac{\Pi _{n,i}}{2^i}\right) ({b-1})^i \Pi _{n,i}. \end{aligned}$$

Remark 1

Note that the lower bounds $\mathcal {LBV}^\mathcal {O}$ are based on the column balance of d. It is useful for assessing the uniformity of an orthogonal array.

5 Optimal Strategy Structure and Its Implementation

In this section, a closer look at the optimal strategy structure and its implementation for this problem with a comparison study between it and the mentioned strategies (cf. Sect. 2) is given. The foregoing analysis provided new and efficient analytical expressions and lower bounds of the uniformity criteria measured by ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}.$ The forthcoming discussions provides new and efficient analytical expressions and lower bounds for these discrepancies for symmetric two-level added r runs designs $\mathcal {A}\in \mathbb {N}(r; 2^m)$ and extended designs $\mathcal {E}\in \mathbb {E}(n+r; 2^m)$, which can be used as benchmarks for constructing optimal extended designs. Using the new formulations of the uniformity criteria and the new lower bounds as benchmarks, we can obtain two-level extended designs with low discrepancy. In order to present the main results, we state and prove, wherever necessary, the following lemmas, remarks and theorems.

Theorem 5

For any symmetric two-level extended design $\mathcal {E}\in \mathbb {E}(n+r; 2^m)$, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {E})]^2= & {} \frac{2nrc}{(n+r)^2}+\left( \frac{n}{n+r}\right) ^2[{\mathcal {VD}}(\mathcal {B})]^2+\left( \frac{r}{n+r}\right) ^2[{\mathcal {VD}}(\mathcal {A})]^2 \nonumber \\&+\frac{2a^m}{(n+r)^2}\left[ \sum _{i=1}^{n}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}\right] , \end{aligned}$$

(5.1)

where $a,~b,~c,~[{\mathcal {VD}}(\mathcal {B})]^2$ and $[{\mathcal {VD}}(\mathcal {A})]^2$ as in Sect. 3.

Proof

We only prove this theorem following Theorem 3, and the other two forms given in Theorems 1 and 2 are similar. Following Theorem 1 in Elsawah and Qin [8], we can write the uniformity criteria based on the symmetric two-level extended designs as follows

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {E})]^2&=c+\frac{a^m}{(n+r)^2}\left[ \sum _{i=1}^{n+r}\sum _{j=1}^{n+r}b^{\zeta _{ij}}\right] \nonumber \\&=\left[ \frac{n^2+r^2+2nr}{(n+r)^2}\right] c+\frac{a^m}{(n+r)^2}\left[ \sum _{i=1}^{n}\sum _{j=1}^{n}b^{\zeta _{ij}}\right. \\&\quad \left. +\sum _{i=n+1}^{n+r}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}+2\sum _{i=1}^{n}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}\right] \nonumber \\&=\left[ \frac{2nr}{(n+r)^2}\right] c+\left( \frac{n^2}{(n+r)^2}\right) \left[ c+\frac{a^m}{n^{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}b^{\zeta _{ij}}\right] \nonumber \\&~~+\left( \frac{r^2}{(n+r)^2}\right) \left[ c+\frac{a^m}{r^{2}}\sum _{i=n+1}^{n+r}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}\right] +\frac{2a^m}{(n+r)^2}\left[ \sum _{i=1}^{n}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}\right] . \end{aligned}$$

Now, from (3.3) the proof is obvious (cf. [10]). $\square$

Remark 2

The extended design $\mathcal {E} \in \mathbb {E}(n+r; 2^m)$ is the design consisting of the treatment combinations belonging to both $\mathcal {B}\in \mathbb {B}(n; 2^m)$ and $\mathcal {A}\in \mathbb {N}(r; 2^m).$ It is to be noted that, if we define $n=2\ell _1$ and $r=2\ell _2+u,~u=0,1,$ then for each factor of the extended design $\mathcal {E},$$2-u$ levels occur $\ell _1+\ell _2$ times and the remaining u levels appear $\ell _1+\ell _2+1$ times.

Lemma 2

For any symmetric two-level nearly balanced design $\mathcal {N} \in \mathbb {N}(r; 2^m),$ let $r=2\ell _2+u,~u=0,1$ and $\zeta _{ij}=\sharp \left\{ (i,j): x_{ik}=x_{jk}, 1\le k \le m\right\} ,$ where $\sharp \left\{ \varphi \right\}$ is the cardinality of the set $\varphi$. Then, we have

$$\begin{aligned} \sum _{i=1}^{r}\sum _{j\ne i}^{r}\zeta _{ij}=2{m\ell _2(\ell _2+u-1)},~\zeta _{ii}=m. \end{aligned}$$

Lemma 3

For any symmetric two-level extended design $\mathcal {E}\in \mathbb {E}(n+r; 2^m)$, let $n=2\ell _1$ and $\zeta _{ij}$ as in Lemma  2. Then, we have

$$\begin{aligned} \sum _{i=1}^{n}\sum _{j=n+1}^{n+r}\zeta _{ij}=mr\ell _1. \end{aligned}$$

In view of Lemma 2, Corollary 1 and Theorem 4, we can get the following new lower bounds for the uniformity criteria measured by ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ based on the added design $\mathcal {A} \in \mathbb {N}(r;2^{m})$$([{\mathcal {VD}}(\mathcal {A})]^2).$

Theorem 6

For any symmetric two-level added design $\mathcal {A} \in \mathbb {N}(r;2^{m}),$ let $r=2\ell _2+u,~u=0,1,~r\ge 2,$$\tilde{f}_j$ is the integral pert of $\frac{r}{2^j},~\tilde{z}_j=r-2^j\tilde{f}_j,~\tilde{\theta }=\frac{2{m\ell _2(\ell _2+u-1)}}{r(r-1)},~\tilde{\alpha }+\tilde{\beta }=r(r-1),~\tilde{\alpha } \tilde{\zeta }+\tilde{\beta }(\tilde{\zeta }+1)=2{m\ell _2(\ell _2+u-1)},$$\tilde{\zeta }$ and $\tilde{\lambda }$ are the integral part and fractional part of $\tilde{\theta }$, respectively, and $\Pi _{r,i}$ is the residual of $r (mod ~2^i).$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {A})]^2\ge \mathcal {LBV}_{\mathcal {A}}=\max \left\{ \mathcal {LBV}_{1\mathcal {A}}, \mathcal {LBV}_{2\mathcal {A}}, \mathcal {LBV}_{3\mathcal {A}}, \mathcal {LBV}^\mathcal {O}_{4\mathcal {A}}\right\} , \end{aligned}$$

where

$$\begin{aligned}&\mathcal {LBV}_{1\mathcal {A}}=c+\frac{a^m}{r^{2}}\sum _{j=0}^{m}\left( \begin{array} {c} m \\ j \end{array} \right) \left( b-1\right) ^j \left[ r\tilde{f}_j+\tilde{z}_j(\tilde{f}_j+1)\right] ,\\&\mathcal {LBV}_{2\mathcal {A}}=c+\frac{a^mb^m}{r}+\frac{(r-1)(1+\tilde{\lambda }(b-1))}{ r}a^mb^{\tilde{\zeta }},\\&\mathcal {LBV}_{3\mathcal {A}}=c+\frac{a^mb^m}{r}+\frac{a^mb^{\tilde{\zeta }}}{r^2}(\tilde{\alpha } +\tilde{\beta } b) \end{aligned}$$

and

$$\begin{aligned} \mathcal {LBV}^\mathcal {O}_{4\mathcal {A}}=c+a^m\left( \frac{1+b}{2}\right) ^m+\frac{a^m}{r^2}\sum _{i=1}^m \left( \begin{array} {c} m \\ i \end{array} \right) \left( 1-\frac{\Pi _{r,i}}{2^i}\right) ({b-1})^i \Pi _{r,i}. \end{aligned}$$

Remark 3

As a special case of Theorem 6, when the added design $\mathcal {A}$ is a balanced design (i.e., r is even), then the results of Theorem 6 are the same as those in Corollary 1, i.e., $\mathcal {LBV}_\mathcal {A}=\mathcal {LBV}_\mathcal {B}.$

Remark 4

When $r=1,$ the lower bounds of the uniformity criteria based on the added design $\mathcal {A} \in \mathbb {N}(1;2^{m})$ given in Theorem  6 will be as follows (cf. Theorem 3)

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {A})]^2=\mathcal {LBV}_{\mathcal {A}}=c+a^mb^m. \end{aligned}$$

Now, in view of Lemma 1 and Lemma 4 in Elsawah and Qin [6], we can get the following lower bound for $\sum _{i=1}^{n}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}$ in terms of the uniformity criteria.

Corollary 2

For any symmetric two-level extended design $\mathcal {E}\in \mathbb {E}(n+r;2^{m}),$ let $n=2\ell _1,~\rho +\delta =nr,~\rho \mu +\delta (\mu +1)=mr\ell _1$ and $\mu$ is the integer part of $\frac{m}{2}.$ Then, we have

$$\begin{aligned} \sum _{i=1}^{n}\sum _{j=n+1}^{n+r}b^{\zeta _{ij}}\ge b^{\mu }(\rho +\delta b). \end{aligned}$$

From Theorems 5 and 6 and Corollary 2, we can get the following lower bounds for the uniformity criteria measured by ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ based on the two-level extended design $\mathcal {E}$$([\mathcal {VD}(\mathcal {E})]^2).$

Theorem 7

For any symmetric two-level extended design $\mathcal {E}\in \mathbb {E}(n+r; 2^m)$ and $r\ge 2,$ let $n=2\ell _1,~\rho +\delta =nr,~\rho \mu +\delta (\mu +1)=mr\ell _1$ and $\mu$ is the integral part of $\frac{m}{2}.$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {E})]^2\ge {{\mathcal {LBV}_\mathcal {E}}}, \end{aligned}$$

where

$$\begin{aligned} {{\mathcal {LBV}_{\mathcal {E}}}}= & {} \frac{2nrc}{(n+r)^2}+\left( \frac{n}{n+r}\right) ^2[{\mathcal {VD}}(\mathcal {B})]^2+\left( \frac{r}{n+r}\right) ^2\mathcal {LBV}_{\mathcal {A}}\\&+\frac{2a^m}{(n+r)^2} b^{\mu }(\rho +\delta b). \end{aligned}$$

Remark 5

In view of Remark 4 when $r=1,$ the lower bounds for the extended design $\mathcal {E}\in \mathbb {E}(n+1;2^{m})$ given in Theorem 7 will be as follows

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {E})]^2\ge {{\mathcal {LBV}_\mathcal {E}}}, \end{aligned}$$

where

$$\begin{aligned} {{\mathcal {LBV}_{\mathcal {E}}}}= & {} \frac{2nc}{(n+1)^2}+\left( \frac{n}{n+1}\right) ^2[{\mathcal {VD}}(\mathcal {B})]^2+\left( \frac{1}{n+1}\right) ^2\left[ c+a^mb^m\right] \\&+\frac{2a^m}{(n+1)^2}b^{\mu }(\rho +\delta b). \end{aligned}$$

We will close this section with the following important discussion, which would give a comparison study between the second possible strategy and our new strategy as well as help the reader better understand the value of our new strategy.

According to the second possible strategy, if we ignore the extended structure (our method) of the extended design, then the extended design can be regarded as a balanced or nearly balanced design upon whether $n+r$ (r) is even or odd, respectively, $\mathcal {E}\in \mathbb {N}(n+r;2^{m}).$ Hence, from Theorem 6, the lower bounds of $[\mathcal {VD}(\mathcal {E})]^2$ can be obtained as follows.

Theorem 8

For any symmetric two-level nearly balanced design $\mathcal {N}\in \mathbb {N}(n+r;2^{m}),$ let $n+r=2\ell +u,~u=0,1,~\ell =\ell _2+\ell _1,~\omega =\frac{2{m\ell (\ell +u-1)}}{(n+r)(n+r-1)},~\tau =\frac{n+r}{2^j},$$\eta _j$ is the integral part of $\tau ,~\delta _j=n+r-2^j\eta _j,~\lambda _1+\lambda _2=(n+r)(n+r-1)$ and $\lambda _1 \xi +\lambda _2(\xi +1)=2{m\ell (\ell +u-1)},$$\xi$ and $\sigma$ are the integral part and fractional part of $\omega$, respectively, and $\Pi _{n+r,i}$ is the residual of $(n+r) (mod~2^i).$ Then, we have

$$\begin{aligned}{}[{\mathcal {VD}}(\mathcal {N})]^2\ge \mathcal {LBV}_{u}=\max \left\{ \mathcal {LBV}_{1u},~\mathcal {LBV}_{2u},~\mathcal {LBV}_{3u},\mathcal {LBV}^\mathcal {O}_{4u}\right\} , \end{aligned}$$

where

$$\begin{aligned}&{\mathcal {LBV}_{1u}}=c+\frac{a^m}{(n+r)^{2}}\sum _{j=0}^{m}\left( \begin{array} {c} m \\ j \end{array} \right) \left( b-1\right) ^j \left[ (n+r)\eta _j+\delta _j(\eta _j+1)\right] ,\\&{\mathcal {LBV}_{2u}}=c+\frac{a^mb^m}{n+r}+\frac{(n+r-1)(1+\sigma (b-1))}{(n+r)}a^mb^{\xi },\\&{{\mathcal {LBV}_{3u}}}=c+\frac{a^mb^m}{n+r}+\frac{a^mb^{\xi }}{(n+r)^2}(\lambda _1 +\lambda _2 b) \end{aligned}$$

and

$$\begin{aligned} \mathcal {LBV}^\mathcal {O}_{4u}=c+a^m\left( \frac{1+b}{2}\right) ^m+\frac{a^m}{(n+r)^2}\sum _{i=1}^m \left( \begin{array} {c} m \\ i \end{array} \right) \left( 1-\frac{\Pi _{n+r,i}}{2^i}\right) ({b-1})^i \Pi _{n+r,i}. \end{aligned}$$

Remark 6

Comparing to designs in $\mathbb {N}(n+r;2^{m}),$ the most obvious advantage of the extended design $\mathcal {E}$ in $\mathbb {E}(n+r; 2^m)$ is its extended structure and maintaining better uniformity, where numerical examples (see Examples 1, 2 in Sect. 6) show that the lower bounds ${{\mathcal {LBV}_\mathcal {E}}}$ given in Theorem 7 under our new strategy are tighter than the lower bounds ${{\mathcal {LBV}_u}}$ given in Theorem 8 under the second possible strategy for any uniformity criteria given above at all values of n,  m and r which means that our new strategy performs better than the second possible strategy.

6 Searching for Optimal Extended Designs with Examples

Optimal extended design An extended design $\mathcal {E}^*\in \mathbb {E} (n+r; 2^{m})$ for given a uniform (nearly uniform) design $\mathcal {B}\in \mathbb {B} (n; 2^{m})$ is optimal with respect to one of the above uniformity criteria if and only if for any extended design $\mathcal {E}\in \mathbb {E} (n+r; 2^{m})$ for given uniform (nearly uniform) design $\mathcal {B}\in \mathbb {B} (n; 2^{m}),$ we have

$$\begin{aligned} ([{\mathcal {VD}}(\mathcal {E}^*)]^2) \le ([{\mathcal {VD}}(\mathcal {E})]^2). \end{aligned}$$

Furthermore, the design $\mathcal {A}$ under which we get the optimal extended design for given $\mathcal {B}$ is called an optimal added design, denoted as $\mathcal {A}^*.$

Uniform design and stopping the searching process Previous lower bounds of the uniformity criteria values $(\mathcal {LBV}_\mathcal {E})$ for two-level extended designs based on various discrepancies $([{\mathcal {VD}}(\mathcal {E})]^2)$ given in Theorem 7 can serve as benchmarks for searching for uniform extended designs. When we search for the uniform extended design, if the uniformity criterion value achieves the lower bound of one of the above criteria ($\mathcal {LBV}_\mathcal {E})$, the searching process could be stopped and the current design is a uniform design with respect to this criterion. In some circumstances, the lower bounds $\mathcal {LBV}_\mathcal {E}$ cannot be reached and therefore are conservative. Then, by analogy with D-efficiency under point or interval estimation, the efficiency of a design $\mathcal {E} \in \mathbb {E} (n+r; 2^{m})$ with regard to the uniformity criteria may be defined as a ratio

$$\begin{aligned} \digamma =\frac{\mathcal {LBV}_\mathcal {E}}{[{\mathcal {VD}}(\mathcal {E})]^2}. \end{aligned}$$

When the ratio $\digamma =1,$ the design $\mathcal {E}$ is called a uniform design, but when the ratio $\digamma$ is close to 1(0.95,  for example), the design $\mathcal {E}$ is called a nearly uniform design, i.e., when $\digamma$ is closer to 1, the design is more uniform.

In the remainder of this section, illustrative examples are provided where numerical studies lend further support to our theoretical results. In the following examples, for the given uniform or nearly uniform design $\mathcal {B},$ we give its ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ values, denoted by ${\mathcal {LD}}(\mathcal {B}),$${\mathcal {SD}}(\mathcal {B}),$${\mathcal {WD}}(\mathcal {B}),$${\mathcal {CD}}(\mathcal {B})$ and ${\mathcal {MD}}(\mathcal {B}),$ with lower bounds $\mathcal {LBL}_\mathcal {B},$${\mathcal {LBS}}_\mathcal {B},$${\mathcal {LBW}}_\mathcal {B},$${\mathcal {LBC}}_\mathcal {B}$ and ${\mathcal {LBM}}_\mathcal {B}$ given in Corollary 1. On the other hand, for the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  $\mathcal {A^*}')',$ we give its ${\mathcal {LD}}$, ${\mathcal {SD}}$, ${\mathcal {WD}},$${\mathcal {CD}}$ and ${\mathcal {MD}}$ values, denoted by ${\mathcal {LD}}(\mathcal {E}^*),$${\mathcal {SD}}(\mathcal {E}^*),$${\mathcal {WD}}(\mathcal {E}^*),$${\mathcal {CD}}(\mathcal {E}^*)$ and ${\mathcal {MD}}(\mathcal {E}^*),$ given in Theorem 5 with lower bounds ${\mathcal {LBL}}_{\mathcal {E}},$${\mathcal {LBS}}_{\mathcal {E}},$${\mathcal {LBW}}_{\mathcal {E}},$${\mathcal {LBC}}_{\mathcal {E}}$ and ${\mathcal {LBM}}_{\mathcal {E}},$ given in Theorem 7 (Remark 4, when $r=1$), respectively. Finally, in view of Theorem 8 the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}} ,~{{\mathcal {LBS}_u}},~{{\mathcal {LBW}_u}},~{{\mathcal {LBC}_u}}$ and ${{\mathcal {LBM}_u}},$ which are smaller than the lower bounds using our extended structure method given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively, in the following examples.

Example 1

Consider the following symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(4; 2^{7})$ given below.

$$\begin{aligned}\mathcal {B}=\left[ \begin{array}{ccccccc} 0 &{} 0 &{} 0 &{} 1 &{} 1 &{} 1 &{} 1\\ 1 &{} 0 &{} 1 &{} 1 &{} 0 &{} 0 &{} 0\\ 1 &{} 1 &{} 0 &{} 0 &{} 1 &{} 1 &{} 0\\ 0 &{} 1 &{} 1 &{} 0 &{} 0 &{} 0 &{} 1\\ \end{array} \right] . \end{aligned}$$

From the numerical results in Table 2, we get

• $\mathcal {LBL}_\mathcal {B}=[{\mathcal {LD}}(\mathcal {B})]^2=0.1478,~ \mathcal {LBS}_\mathcal {B}=[{\mathcal {SD}}(\mathcal {B})]^2=24.9071,~\mathcal {LBW}_\mathcal {B}=[{\mathcal {WD}}(\mathcal {B})]^2=2.2731,~\mathcal {LBC}_\mathcal {B}=[{\mathcal {CD}}(\mathcal {B})]^2= 0.4678$ and $\mathcal {LBM}_\mathcal {B}=[{\mathcal {MD}}(\mathcal {B})]^2= 4.8706.$ The efficiency of the design $\mathcal {B}$ under the various discrepancies is $\digamma =1,$ which means that $\mathcal {B}$ is a uniform design with respect to the various discrepancies.

• [I] Adding the following run

  $$\begin{aligned} \mathcal {A}^*=[1 ~~~ 1 ~~~ 0 ~~ ~ 1 ~ ~~ 0 ~~ ~ 1 ~ ~~ 1] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(5; 2^{7}).$ From the numerical results in Table 2, we get

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2= 0.1165 ,~ \mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2= 20.9071 ,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2=2.3417,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2= 0.4755$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2=5.1351.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}=0.1115,~{{\mathcal {LBS}_u}}=20.2671,~{{\mathcal {LBW}_u}}=2.2977,~{{\mathcal {LBC}_u}}=0.4598$ and ${{\mathcal {LBM}_u}}=5.0318,$ which are smaller than the lower bounds using our extended structure method given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

• [II] Adding the following two runs

  $$\begin{aligned} \mathcal {A}^*=\left[ \begin{array}{ccccccc}1&{} 1&{} 0 &{}1&{} 0&{} 1&{} 1\\ 0 &{}0&{} 1&{} 0&{} 1&{} 0&{} 0\\ \end{array} \right] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(6; 2^{7}).$ From the numerical results in Table 2, we get

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2=0.0892,~\mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2=17.4071,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2=2.0908,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2= 0.3972$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2=4.4663.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}= 0.0801,~{{\mathcal {LBS}_u}}= 16.2405,~{{\mathcal {LBW}_u}}=1.9937, ~{{\mathcal {LBC}_u}}=0.3634$ and ${{\mathcal {LBM}_u}}=4.2348,$ which are smaller than the lower bounds using our extended structure method given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

• [III] Adding the following four runs

  $$\begin{aligned}\mathcal {A}^*=\left[ \begin{array}{ccccccc} 1&{} 1&{} 1&{} 1 &{}1&{} 1&{} 1\\ 0&{} 1&{} 0 &{}1 &{}0 &{}0&{} 0\\ 0 &{}0&{} 1&{} 0&{} 1 &{}1 &{}0\\ 1&{} 0&{} 0&{} 0 &{}0&{} 0 &{}1\\ \end{array} \right] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(8; 2^{7}).$ From the numerical results in Table 2, we get

  Table 2 Numerical results of Example 1

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2= 0.0540 ,~ \mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2=12.9071 ,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2= 1.9226 ,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2=0.3356$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2=4.0784.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}= 0.0462,~{{\mathcal {LBS}_u}}= 11.9071,~{{\mathcal {LBW}_u}}=1.8540, ~{{\mathcal {LBC}_u}}=0.3112$ and ${{\mathcal {LBM}_u}}=3.9169 ,$ which are smaller than the lower bounds using our extended structure given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

Example 2

Consider the following symmetric two-level balanced design $\mathcal {B}\in \mathbb {B}(8; 2^{13})$ given below.

$$\begin{aligned} \mathcal {B}=\left[ \begin{array}{ccccccccccccc} 1&{} 1 &{}1&{} 1&{} 1&{} 1&{} 1&{} 1&{} 1&{} 1&{} 1&{} 1&{} 1\\ 1&{} 1&{} 1&{} 0&{} 0&{} 1&{} 0&{} 0 &{}0 &{}1&{} 0&{} 0&{} 1\\ 1&{} 0&{} 0&{} 0&{} 1&{} 0&{} 1&{} 0&{} 1&{} 0&{} 0&{} 1&{} 1\\ 1&{} 0&{} 1&{} 1&{} 1&{} 0&{} 0&{} 0&{} 0&{} 0&{} 1&{} 0&{} 0\\ 0&{} 1&{} 0&{} 1&{} 0&{} 0&{} 0&{} 1&{} 1&{} 0&{} 0&{} 0&{} 1\\ 0 &{}1 &{}0 &{}0&{} 1&{} 0&{} 0&{} 1&{} 0&{} 1&{} 1&{} 1&{} 0\\ 0&{} 0&{} 1&{} 0&{} 0&{} 1&{} 1&{} 1&{} 1&{} 0&{} 1&{} 0&{} 0\\ 0 &{}0 &{}0 &{}1&{} 0&{} 1&{} 1&{} 0&{} 0&{} 1&{} 0&{} 1&{} 0\\ \end{array} \right] . \end{aligned}$$

From the numerical results in Table 3, we get

• $\mathcal {LBL}_\mathcal {B}=[{\mathcal {LD}}(\mathcal {B})]^2= 0.1066,~ \mathcal {LBS}_\mathcal {B}=[{\mathcal {SD}}(\mathcal {B})]^2=984.4995 ,~\mathcal {LBW}_\mathcal {B}=[{\mathcal {WD}}(\mathcal {B})]^2= 26.3658,~\mathcal {LBC}_\mathcal {B}=[{\mathcal {CD}}(\mathcal {B})]^2= 1.7447$ and $\mathcal {LBM}_\mathcal {B}=[{\mathcal {MD}}(\mathcal {B})]^2= 156.2616.$ The efficiency of the design $\mathcal {B}$ under the various discrepancies is $\digamma =1,$ which means that $\mathcal {B}$ is a uniform design with respect to the various discrepancies.

• [I] Adding the following run

  $$\begin{aligned} \mathcal {A}^*=\left[ \begin{array}{ccccccccccccc}1&1&1&1&0&0&1&1&0&0&0&1&0\end{array} \right] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(9; 2^{13}).$ From the numerical results in Table 3, we get

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2= 0.0939 ,~ \mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2= 880.4501 ,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2= 26.2025,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2= 1.6993$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2=156.6216.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}=0.0935,~{{\mathcal {LBS}_u}}= 877.2896,~{{\mathcal {LBW}_u}}=26.0236,~{{\mathcal {LBC}_u}}=1.6804$ and ${{\mathcal {LBM}_u}}=155.4676,$ which are smaller than the lower bounds using our extended structure given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

• [II] Adding the following two runs

  $$\begin{aligned} \mathcal {A}^*=\left[ \begin{array}{ccccccccccccc}1&{} 1 &{}1&{} 1&{} 0&{} 0 &{}1 &{}1&{} 0&{} 0&{} 0&{} 1&{} 0\\ 0&{} 0 &{}0&{} 0&{} 1&{} 1 &{}0 &{}0&{} 1&{}1&{} 1&{} 0&{}1\\ \end{array} \right] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(10; 2^{13}).$ From the numerical results in Table 3, we get

  Table 3 Numerical results of Example 2

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2= 0.0834 ,~ \mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2= 794.5995 ,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2=25.0958,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2= 1.5846$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2=149.0653.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}=0.0825,~{{\mathcal {LBS}_u}}= 786.8995,~{{\mathcal {LBW}_u}}=24.4423,~{{\mathcal {LBC}_u}}=1.5188$ and ${{\mathcal {LBM}_u}}=144.7055,$ which are smaller than the lower bounds using our extended structure given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

• [III] Adding the following four runs

  $$\begin{aligned} \mathcal {A}^*=\left[ \begin{array}{ccccccccccccc}1&{} 1 &{}1&{} 1&{} 0&{} 0 &{}1 &{}1&{} 0&{} 0&{} 0&{} 1&{} 0\\ 1&{} 0 &{}0&{} 1&{} 1&{} 1 &{}0 &{}1&{} 1&{}1&{} 0&{} 0&{}0\\ 0&{} 1 &{}0&{} 0 &{}1 &{}1 &{}1&{} 0 &{}0 &{}0 &{}1 &{}0&{} 1\\ 0 &{}0&{} 1&{} 0 &{}0 &{}0&{} 0&{} 0 &{}1 &{}1&{} 1&{} 1&{} 1\\ \end{array} \right] \end{aligned}$$

  to $\mathcal {B}$ gives the optimal extended design $\mathcal {E}^*=(\mathcal {B}'$  ${\mathcal {A}^*}')'\in \mathbb {E}(12; 2^{13}).$ From the numerical results in Table 3, we get

  • $\mathcal {LBL}_{\mathcal {E}}=[{\mathcal {LD}}(\mathcal {E}^*)]^2=0.0674 ,~ \mathcal {LBS}_{\mathcal {E}}=[{\mathcal {SD}}(\mathcal {E}^*)]^2= 663.1662 ,~\mathcal {LBW}_{\mathcal {E}}=[{\mathcal {WD}}(\mathcal {E}^*)]^2= 23.6462,~\mathcal {LBC}_{\mathcal {E}}=[{\mathcal {CD}}(\mathcal {E}^*)]^2= 1.4191$ and $\mathcal {LBM}_{\mathcal {E}}=[{\mathcal {MD}}(\mathcal {E}^*)]^2= 140.1545.$ The efficiency of the design $\mathcal {E}^*$ under the various discrepancies is $\digamma =1,$ which means that the optimal extended design $\mathcal {E}^*$ is a uniform design with respect to the various discrepancies.

  • In view of Theorem 8, the lower bounds of ${\mathcal {LD}},~{\mathcal {SD}},~{\mathcal {WD}},~{\mathcal {CD}}$ and ${\mathcal {MD}}$ are, respectively, ${{\mathcal {LBL}_u}}= 0.0664,~{{\mathcal {LBS}_u}}= 655.1662,~{{\mathcal {LBW}_u}}= 23.1600, ~{{\mathcal {LBC}_u}}= 1.3683$ and ${{\mathcal {LBM}_u}}= 137.0015,$ which are smaller than the lower bounds using our extended structure given in Theorem 7, $\mathcal {LBL}_{\mathcal {E}},~\mathcal {LBS}_{\mathcal {E}},~\mathcal {LBW}_{\mathcal {E}},~\mathcal {LBC}_{\mathcal {E}}$ and $\mathcal {LBM}_{\mathcal {E}},$ respectively.

Table 4 Numerical results of Example 3

Example 3

Consider the design $\mathcal {B}\in \mathbb {B}(4; 2^{7})$ given in Example 1. Adding the following four runs

$$\begin{aligned} \mathcal {A}=\left[ \begin{array}{ccccccc} 1 &{}1 &{}1&{} 0&{} 0&{} 1&{} 0\\ 0&{} 1&{} 0&{} 0&{} 1 &{}0&{} 1\\ 0 &{}0 &{}1 &{}1 &{}0&{} 1&{} 1\\ 1 &{}0&{} 0&{} 1&{} 1 &{}0&{} 0\\ \end{array} \right] \end{aligned}$$

to $\mathcal {B}$ gives the extended design $\mathcal {E}=(\mathcal {B}'$  ${\mathcal {A}}')'\in \mathbb {E}(8; 2^{7}).$ From Table 4, we can show that the efficiency of the given (original) design $\mathcal {B}$ under the various discrepancies is $\digamma =1,$ which means that the given (original) design is a uniform design with respect to the various discrepancies. The efficiency of the added design $\mathcal {A}$ under the various discrepancies is $\digamma =1,$ which means that the added design is a uniform design with respect to the various discrepancies. However, the extended design $\mathcal {E}=(\mathcal {B}'$  ${\mathcal {A}}')'$ is not a uniform or a nearly uniform design which means that the third possible strategy is not an effective strategy. See the third possible strategy in Sect. 2.

Remark 7

According to Examples 1 and 2, we can show that our new strategy performs better than the second possible strategy, and according to Example 3, we get that the third possible strategy is not an effective strategy as well as it is easy to check that the first possible strategy is not a feasible strategy. Thus, our strategy is the most effective.

7 Conclusions and Possible Extensions

This paper considers the study of the optimality of the extended design generated by adding a few runs to an existing two-level uniform design. The criteria of the optimality of additional runs are the uniformity criteria measured by Lee, symmetric, wrap-around, centered and mixture discrepancy. For symmetric two-level factorials as the original designs, we investigate new analytical expressions of the above discrepancies for extended designs. We derive results connecting uniformity and orthogonality and show that these criteria agree quite well, which provide further justifiable interpretation for two measures of orthogonality by the consideration of uniformity. This analysis provides new lower bounds for these discrepancies for symmetric two-level extended designs, which can be used as benchmarks for searching for optimal additional runs. Illustrative examples are provided, where numerical studies lend further support to our theoretical results. Our results provide a theoretical justification for the optimal addition of runs to uniform designs in terms of uniformity criteria.

In this paper, we discuss in depth all the possible strategies of this problem with a comparison study and provide the optimal strategy. As Remark 7 pointed out we can show that our new strategy performs better than the second possible strategy and the third possible strategy is not an effective strategy as well as it is easy to check that the first possible strategy is not a feasible strategy. Therefore, our strategy is the best one. On the other hand, from our numerical results we can show that for many uniform or nearly uniform designs $\mathcal {B}\in \mathbb {B}(n; 2^m)$, the designs $\mathcal {B}$ has the same optimal added designs (extended) $\mathcal {A}^*~ (\mathcal {E}^*)$ under the various discrepancies.

For implementing the uniform design in industrial, engineering, economy and computer science, experiments, the most important step is to choose a suitable uniform design table to accommodate the number of factors and levels. This can now easily be done by using the new formulations and lower bounds of the uniformity criteria measured by Lee, symmetric, wrap-around, centered and mixture discrepancies in this paper as the benchmark, we can construct two-level extended designs with low discrepancy.

Finally, we briefly indicate some promising recent topics in the construction of optimal extended designs that are not covered in this paper. They are still undergoing rapid development by the authors of this paper and hence are yet to crystallize. We obtained some simulation results for some of the following problems, but we cannot give any conclusion at this stage. There are many open problems in this area, and we mention some problems worth to study further.

• Our results for constructing optimal extended designs in this paper only concentrate on specific types of symmetric factorials with two levels. Then, future research will concentrate on generalizing our results to the case of general multilevel designs as well as flexible levels designs.

• Future research will concentrate on generalizing our results for constructing uniform or nearly uniform designs with large numbers of runs and/or large numbers of factors.

• Since advantages of our new construction method have been revealed, we can use it to update uniform (nearly uniform) design tables in view of all the discrepancies. We are curious about the difference and similarity between the new designs (extended designs) and the existing uniform designs (nearly uniform designs) in different categories.