Some Considerations in Optimal Response Surface Designs

Abstract

Second-order design of experiments over the cuboidal region is considered for the situation where primary interest of the experimenter lies in estimating parameters corresponding to second-degree terms in the model. The D-optimal designs for the situation are derived. Relative performances of the derived designs and D-optimal designs for all parameters are investigated. It is found that the derived D-optimal designs have high efficiency and may be preferable to the latter. Optimal designs among the symmetric product designs and the central composite designs are also studied for the set-up under consideration.

1 Introduction

Consider the standard linear model set up in which $E\left\{{Y}_{{\varvec{x}}}\right\}={f}^{t}\left(x\right)\theta$ = ${f}_{1}^{t}\left(x\right){\theta }_{1}+{f}_{2}^{t}\left(x\right){\theta }_{2}$ where ${Y}_{{\varvec{x}}}$ is the value of the response variable Y at $x=({x}_{1}, \dots , {x}_{k})$ the value of the vector of explanatory variables, ${f}^{t}\left(x\right)=$$\left({f}_{1}^{t}\left(x\right),{f}_{2}^{t}\left(x\right)\right)$is a row vector of p linearly independent functions of $x$ and $\theta ={\left({\theta }_{1}^{t},\boldsymbol{ }{\theta }_{2}^{t}\right)}^{t}$ is a column vector of p parameters. Let $\Omega =\left\{x\right\}$, the experimental region or the design space, be closed and compact. Any experimental design ξ is then a probability measure on $\Omega$. An exact or N-observation design is thus a discrete probability measure with masses that are integer multiples of ${N}^{-1}$. It is assumed that the observations are independent and homoscedastic with a common variance ${\sigma }^{2}$ which without loss of generality may be assumed to be unity. If N trials are carried out in accordance with ξ then the variance–covariance matrix of the least squares estimators (maximum likelihood estimators under normal theory assumptions) of the parameters $\widehat{\theta }$ is given by ${N}^{-1}{M}^{-1}(\xi )$ where $M\left(\xi \right)={\int }_{\Omega }f\left(x\right){f}^{t}\left(x\right)\xi (\mathrm{d}x)$ is the information matrix of the design ξ. The matrix $M\left(\xi \right)$, necessarily symmetric, may be written as a partitioned matrix

$$ M\left( \xi \right) = \left[ {\begin{array}{*{20}c} {M_{11} \left( \xi \right)}& {M_{12} \left( \xi \right)} \\ {M_{21} \left( \xi \right)} &{M_{22} \left( \xi \right)} \\ \end{array} } \right]$$

where $M_{ij} \left( \xi \right) = \mathop \int \nolimits_{{\Omega }}^{{}} f_{i} \left( x \right)f_{j}^{t} \left( x \right)\xi \left( {{\text{d}}x}\right)\quad \left( {i,j = 1,2}\right).$

It is to be noted that when all parameters of the model are estimated, the information matrix of ξ for ${\theta }_{1}$ and ${\theta }_{2}$ are given by.

$M_{1.2} \left( \xi \right) = M_{11} \left(\xi \right) - M_{12} \left( \xi \right)M_{22}^{ - 1} \left( \xi \right)M_{21} \left( \xi \right)$ and $M_{2.1} \left(\xi \right) = M_{22} \left( \xi \right) - M_{21} \left( \xi \right)\break M_{11}^{ - 1} \left( \xi \right) M_{12} \left( \xi \right)$, respectively.

There are many different criteria for choosing a design. The most popular optimality criteria appear to be the D-optimality and A-optimality. Under A-optimality, the objective is to minimize the average variance of the estimators of the parameter, i.e., to minimize ${\text{trace }}\,M^{ - 1} \left( \xi \right)$. On the other hand, under D-optimality, the objective is to minimize the generalized variance, i.e., to maximize $\det M\left( \xi \right)$.

In many situations, the experimenter is fairly certain about the necessity of the terms of $f_{1} \left( x \right)$ in the model but unsure about the terms of $f_{2} \left( x \right)$. In that case, his strategy might be to estimate all the parameters in the model but to do so with maximum precision for the parameters in $\theta_{2}$. Under D-optimality criterion, this would require maximizing the determinant of $M_{2.1} \left( \xi \right)$. while under A-optimality criterion, minimization of ${\text{tr}}\{M_{2.1} \left( \xi \right){ }\}^{ - 1}$ would be the objective. After estimation he would test the null Hypothesis $H_{0} :\theta_{2}= 0 \,{\text{against the alternative }}H_{a} :\theta_{2} \ne 0$. If the experimenter has limited resources and cannot perform further trials in the light of conclusions from the test, then he may proceed as follows:

• a. If the null hypothesis is accepted, then (1) retain the estimate of $\theta_{1} \,{\text{and\,\,proceed}},$ the information matrix for $\theta_{1}$ remaining $M_{1.2} \left( \xi \right)$ or (2) discard the old estimates and recompute the estimate of $\theta_{1}$fitting the reduced model $E\left\{ {Y_{{\varvec{x}}} } \right\} =f_{1}^{t} \left( x \right)\theta_{1}$ to the data already obtained and use $M_{11} \left( \xi \right)$ as the information matrix for $\theta_{1}$.

• b. If the null hypothesis is rejected, proceed further using his estimate of $\theta = \left({\theta_{1}^{t} ,\user2{ }\theta_{2}^{t} }\right)^{t}$. and use the information matrix $M\left( \xi \right)$.

A question that naturally arises is how well the above strategy pays in the three situations described. In case (a) the design performance has to be compared to the optimal design for the reduced model while in case (b) the comparison has to be with the optimal design for the full model.

In regression experiments involving quantitative variables, the full model, i.e.,$f\left( x \right)$ may correspond to a full d-th order polynomial in the k variables and the reduced model represented by $f_{1} \left( x \right)$ may be a full polynomial of order $d_{1} ( <d)$ with special interest in the case $d_{1} = d -1$.

In what follows, the above problem is investigated in the context of regression over hypercubic regions with the full model being a second order polynomial (d = 2) and the reduced model being a first-order model ($d_{1} =1)$ in k variables. The design criteria used is D-optimality.

Section 2 provides the optimization problem set-up for second-order models. Section 3 gives derivation of the optimal designs. In Sect. 4, efficiencies of the designs are investigated. In Sect. 5 optimal designs within the class of product designs are derived. Section 6 provides the optimal designs within the class of central composite designs, a class most commonly used by experimenters. Finally, Sect. 7 provides some comments and discussion.

2 Second-Order Model Over Hypercubic Region

Let the design space ${\Omega }$ be a k-dimensional hypercube which without loss of generality may be taken as scaled and centered at the origin, i.e. $\Omega = \left\{ {x = \left( {x_{1} , \ldots , x_{k} } \right); - 1 \le x_{i} \le 1 \qquad \left( {i = 1, \ldots , k} \right)} \right\}.$ Let $f^{t} \left( x \right) = \left( {1, x_{1} ,\ldots , x_{k} ,x_{1}^{2} , \ldots ,x_{k}^{2} ,x_{1} x_{2} , \ldots , x_{k - 1} x_{k} } \right)$ and $f_{1}^{t} \left( x \right) = \left( {1,x_{1} , \ldots , x_{k} }\right)$ so that $f_{2}^{t} \left( x \right) = \left({x_{1}^{2} , \ldots ,x_{k}^{2} ,x_{1} x_{2} , \ldots , x_{k - 1}x_{k} } \right)$. As the optimal designs under A- and D-optimality for polynomial regression models over hypercubic regions are known to be symmetric and permutation invariant consideration needs to be restricted to only such designs [4,5,6]. For any such design $\xi$ all the “odd” moments are zero and one obtains.

$\mathop \smallint \limits_{{\Omega }}^{{}}x_{i} \xi \left( {{\text{d}}x} \right) = \mathop \smallint \limits_{{\Omega }}^{{}} x_{i} x_{j} \xi \left( {{\text{d}}x}\right) = \mathop \smallint \limits_{{\Omega }}^{{}} x_{i}^{3} \xi \left( {{\text{d}}x} \right) = \mathop \smallint \limits_{{\Omega }}^{{}} x_{i} x_{j} x_{l} \xi \left( {{\text{d}}x}\right)$.

$$ = \mathop \smallint \limits_{{\Omega }}^{{}} x_{i}^{2} x_{j} \xi \left({{\text{d}}x} \right) = \mathop \smallint \limits_{{\Omega }}^{{}}x_{i}^{2} x_{j} x_{l} \xi \left( {{\text{d}}x} \right) = \mathop \smallint \limits_{{\Omega }}^{{}} x_{i}^{3} x_{j} \xi \left({{\text{d}}x} \right) = \mathop \smallint \limits_{{\Omega }}^{{}}x_{i} x_{j} x_{l} x_{m} \xi \left( {{\text{d}}x} \right) = 0,$$

$$ \mathop \smallint \limits_{{\Omega }}^{{}}x_{i}^{2} \xi \left( {{\text{d}}x} \right) = \alpha_{2} ,{ }\mathop \smallint \limits_{{\Omega }}^{{}} x_{i}^{4} \xi \left({{\text{d}}x} \right) = \alpha_{4} ,{ }\mathop \smallint \limits_{{\Omega }}^{{}} x_{i}^{2} x_{j}^{2} \xi \left({{\text{d}}x} \right) = \alpha_{22 } ,$$

for $i \ne j \ne l \ne m = 1, \ldots ,k.$

Hence, for a symmetric permutation invariant design $\xi$ the information matrix $M\left( \xi \right)$ has

$$ \begin{aligned} M_{11} \left( \xi \right) & = {\text{diag}}\left\{ {1,\alpha_{2} I_{k} } \right\},M_{22} \left(\xi \right) = {\text{diag}}\left\{ {\left( {\alpha_{4} - \alpha_{22} } \right)I_{k} + \alpha_{22 } J_{k} ,\alpha_{22 } I_{k*} }\right\}, \\ M_{12} \left( \xi \right) & = \left[{\begin{array}{*{20}c} {\alpha_{2} 1_{k}^{t} } & {0_{k*}^{t} } \\ {O_{k} } & O \\ \end{array} } \right]\quad {\text{and}}\quad M_{21}\left( \xi \right) = M_{12}^{t} \left( \xi \right), \end{aligned}$$

(1)

where ${I}_{k}$is the identity matrix of order k, ${J}_{k}$ is the k × k matrix of 1’s, ${1}_{k}^{t}$ is the k-component row vector of 1’s and $0_{k*}^{t}$ is the k*-component row vector of 0’s, $O_{k}$ and $O$ are matrices of 0’s with dimensions k x k and $k \times k^{*}$, respectively where $k^{*} = k\left( {k - 1}\right)/2$.

It follows that for a symmetric permutation invariant design $\xi$,

$$ \begin{aligned} \left| {M_{2.1} \left( \xi \right)} \right| & = \left[ {\left( {\alpha_{4} - \alpha_{22} }\right)^{k - 1} \left( {\alpha_{4} + \left( {k - 1}\right)\alpha_{22} - k\alpha_{2}^{2} } \right)} \right]\alpha_{22}^{{\left( {\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} }\right)}} , \\ \left| {M\left( \xi \right)} \right| & =\alpha_{2}^{k} \left[ {\left( {\alpha_{4} - \alpha_{22} } \right)^{k - 1} \left( {\alpha_{4} + \left( {k - 1} \right)\alpha_{22} -k\alpha_{2}^{2} } \right)} \right]\alpha_{22}^{{\left({\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} } \right)}} . \\\end{aligned} $$

(2)

In the literature it is usual to write $f^{t} \left( x\right) =( 1,x_{1}^{2} , \ldots ,x_{k}^{2} , x_{1} , \ldots ,x_{k} ,x_{1} x_{2}$, $\ldots , x_{k - 1} x_{k})$ and then the corresponding $M\left( \xi \right)$ is given by

$$ M\left( \xi \right) = {\text{diag}}\left\{ {\left[ {\begin{array}{*{20}c} 1 &{\alpha_{2} 1_{k}^{t} } \\ {\alpha_{2} 1_{k} } & {\left( {\alpha_{4}- \alpha_{22} } \right)I_{k} } \\ \end{array} } \right], \alpha_{2}I_{k} , \alpha_{22} I_{k*} } \right\}$$

from which it is much easier to derive $\left| {M\left(\xi \right)} \right|$ and ${\text{tr}}\left( {M^{ - 1} \left( \xi \right)} \right).$ Also $\left| {M_{2.1} \left( \xi \right)}\right|^{ - 1}$ and ${\text{tr}}(M_{2.1}^{ - 1} (\xi ))$ can then readily be obtained by partitioning the matrix $M^{ - 1} (\xi )$ according to the original expression for $f^{t} \left( x \right)$.

We let $\xi_{2}^{D}$and $\xi_{2.1}^{D}$denote the D-optimal designs for the full and reduced models respectively. $\xi_{2}^{D}$ is already known in the literature [1, 2]. In what follows we derive $\xi_{2.1}^{D}$.

3 Optimal Designs for Parameters of Second-Order Terms

If the main purpose is to estimate $\theta_{2}$ with maximum precision, then under D-optimality criterion the objective is to maximize $\left| {M_{{2.1}} \left( \xi \right)}\right| = \left[ {\left( {\alpha _{4} - \alpha _{{22}} }\right)^{{k - 1}} \left( {\alpha _{4} + \left( {k - 1}\right)\alpha _{{22}} - k\alpha _{2}^{2} } \right)} \right]\alpha _{{22}}^{{( {\begin{subarray}{*{20}c} k \\ 2 \\\end{subarray} } )}}$ with respect to the design moments $\alpha_{2} ,\alpha_{4}\,{\text{and}}\,\alpha_{22}$. subject to the constraints $1 \ge \alpha_{2}\ge \alpha_{4} > \alpha_{22} >0$ and $\alpha_{4} +\left( {k - 1} \right)\alpha_{22} - k\alpha_{2}^{2} >0$. The last constraint is due to the non-singularity requirement on $M\left( \xi \right)$ while the other constraints are due to the design region ${\Omega }.$ We consider D-optimality.

3.1 D-Optimal Designs

Clearly the objective function $\left| {M_{2.1} \left( \xi \right)}\right|$ is strictly increasing in $\alpha_{4}$ and since $\alpha_{2} \ge \alpha_{4}$, the D-optimal design $\xi_{2}^{D}$must have $\alpha_{4} =\alpha_{2}$, i.e. the design $\xi_{2.1}^{D}$ (like the design $\xi_{2}^{D})$ must supported at points whose co-ordinates are 0 and ± 1 only. Substituting $\alpha_{4} =\alpha_{2}$, the objective function reduces to

$$ \left| {M_{2.1} \left( \xi \right)} \right|= \left[ {\left( {\alpha_{2} - \alpha_{22} } \right)^{k - 1} \left({\alpha_{2} + \left( {k - 1} \right)\alpha_{22} - k\alpha_{2}^{2} }\right)} \right]\alpha_{22}^{{\left( {\begin{subarray}{*{20}c} k \\2 \\ \end{subarray} } \right)}} .$$

(3)

In order to derive the design $\xi_{2.1}^{D}$, let $\alpha_{2} =u$ and $\alpha_{22} =tu$ where $0 < u,t < 1, ku <1 + \left( {k - 1} \right)t.$ Then after simplifications it can be seen that (3) reduces to

$$ \left| {M_{2.1} \left( \xi \right)} \right|= u^{{\frac{{k\left( {k + 1} \right)}}{2}}} t^{{\frac{{k\left( {k -1} \right)}}{2}}} \left( {1 - t} \right)^{k - 1} \left[ {1 + \left({k - 1} \right)t - ku} \right].$$

(4)

Note that the transformation from $\alpha_{2}$ and $\alpha_{22}$ to $u$ and $t$ is one to one. Hence, we may consider equivalent maximization of the determinant or log of the determinant with respect to $u$ and $t$. Let

$$ f\left( {u,t} \right) = \ln \left| {M_{2.1}\left( \xi \right)} \right| = \frac{{k\left( {k + 1} \right)}}{2}\ln u + \frac{{k\left( {k - 1} \right)}}{2}\ln t + \left( {k - 1}\right)\ln \left( {1 - t} \right) + \ln \left[ {1 + \left( {k - 1}\right)t - ku} \right]. $$

(5)

Equating to zero the partial derivatives of (5) with respect to $u$ and $t$ gives the equations

$$ \frac{{k\left( {k + 1} \right)}}{2u} +\frac{ - k}{{\left[ {1 + \left( {k - 1} \right)t - ku} \right]}} =0, $$

(6)

$$ \frac{{k\left( {k - 1} \right)}}{2t} +\frac{{ - \left( {k - 1} \right)}}{{\left( {1 - t} \right)}} +\frac{{\left( {k - 1} \right)}}{{\left[ {1 + \left( {k - 1} \right)t - ku} \right]}} = 0. $$

(7)

which have to be solved simultaneously. Simplifying (6) gives

$$ u = \frac{{\left( {k + 1} \right)\left[ {1 + \left( {k - 1} \right)t} \right]}}{{k^{2} + k + 2}}.$$

(8)

From (7), we get

$$ u = \frac{{1 + \left( {k - 2} \right)t -\left( {k + 1} \right)t^{2} }}{{k - \left( {k + 2} \right)t}}.$$

(9)

Now, equating (8) and (9) gives

$$ 2\left( {k + 1} \right)t^{2} - \left( {2k -1} \right)t - 1 = 0. $$

(10)

The solutions of (10), are given by $t =\frac{{\left( {2k - 1} \right) \pm \sqrt {\left( { - 2k + 1}\right)^{2} + 8\left( {k + 1} \right)} }}{{4\left( {k + 1}\right)}}.$ But since $0 < t <1$, the positive solution is the only feasible one. Therefore,

$$ t = \frac{{\left( {2k - 1} \right) + \sqrt {4k^{2} + 4k + 9} }}{{4\left( {k + 1} \right)}}.$$

(11)

Now, substituting (11) in (8) or (9) gives

$$ u = \frac{{2k^{2} + k + 5 + \left( {k - 1}\right)\sqrt {4k^{2} + 4k + 9} }}{{4\left( {k^{2} + k + 2}\right)}}. $$

(12)

To check if $u$ and $t$ obtained in (11) and (12), respectively, maximize the determinant of $M_{2.1} \left(\xi \right)$, the determinant of the Jacobian should be $<0$. However, since $u$ (as in (8) and (9)) is not independent of $t$, the method of the Jacobian cannot be applied. Thus, the second derivive $u$ and $t$ have been obtained separately. Since

$$\frac{{\partial^{2} f}}{{\partial u^{2} }}\left( {u,t} \right) =\frac{{ - k\left( {k + 1} \right)}}{{2u^{2} }} + \frac{{ - k^{2}}}{{\left[ {1 + \left( {k - 1} \right)t - ku} \right]^{2} }} < 0 $$

and

$$\frac{{\partial^{2} f}}{{\partial t^{2} }}\left( {u,t} \right) =\frac{{ - k\left( {k - 1} \right)}}{{2t^{2} }} + \frac{{ - \left( {k - 1} \right)}}{{\left( {1 - t} \right)^{2} }} + \frac{{ - \left( {k - 1} \right)^{2} }}{{\left[ {1 + \left( {k - 1} \right)t - ku}\right]^{2} }} < 0, $$

the $D$-optimal design $\xi_{2.1}^{D}$ has $\alpha_{4} = \alpha_{2} =u$ and $\alpha_{22} =tu$ with t and u as given by (11) and (12), respectively. Table 1 displays the numerical values of $u, t,$ and $tu$ after rounding up to four decimal points for the $D$-optimal designs measures $\xi_{2.1}^{D}$ for $k = 2$ up to $k =20.$

Table 1 Non zero moments of $D$-optimal designs measures $\xi_{2.1}^{D}$

It is seen in Table 1 that as $k$ increases, the moments ${\alpha }_{2}(={\alpha }_{4})$ and ${\alpha }_{22}$ increase. Also $\left|{M}_{2}\left({\xi }_{2.1}^{D}\right)\right|$ decreases with increasing k.

3.2 Implementable Designs

We note that both $\xi_{2.1}^{D}$and $\xi_{2}^{D}$are supported at points whose coordinates are 0 and ± 1 only. In our derivations we have so far obtained only the moments of $\xi_{2.1}^{D}$. For actual implementation the design masses that are allocated to the support points must be specified as well. The mass distribution for $\xi_{2}^{D}$is known in the literature. We must derive the mass distribution for $\xi_{2.1}^{D}$. So, consider the set of all points in the design space which may be support points of these optimal designs. These are the $3^{k}$points given by the $3$-level factorial experiment with factor levels $- 1,0,$ and $1$. These $3^{k}$points can be partitioned into $k +1$ sets with the $j{\text{th}}$ set having all points with $j$ nonzero coordinates. The cardinality of the $j$-th set is $\left( {\begin{array}{*{20}c} k \\ j \\\end{array} } \right)2^{j}$$\left( {j = 0, 1, 2, \ldots ,k}\right)$. We need to consider only designs of the type putting positive mass $f_{j}$ uniformly (equally divided) over the points of $j{\text{th}}$ set $\left( {j = 0, 1, 2, \ldots , k}\right).$ For such a design:

$$\alpha_{2} = \alpha_{4} = \frac{1}{k}\mathop \sum \limits_{j =1}^{k} jf_{j} ,\quad \alpha_{22} = \frac{1}{{k\left( {k - 1}\right)}}\mathop \sum \limits_{j = 2}^{k} j\left( {j - 1}\right)f_{j} $$

where $f_{j}$’s are subject to the constraints $f_{j} \ge 0$ $\left( {j = 0,1, \ldots , k} \right)$ and $\mathop \sum \nolimits_{j = 0}^{k} f_{j} =1$ and where $\alpha_{2}$ and $\alpha_{22}$ are corresponding to Table 1. There may be more than one feasible solution to the problem. It is of interest to find solutions that require only three nonzero $f_{j}$’s.These are the ones likely to correspond to designs with a minimum number of support points. A computer program (“Solve” and “End solve”) in “Mathematica” was run to solve the problem numerically and the masses $f_{j}$$\left( {j = 0, 1, \ldots , k}\right)$ obtained are listed in Table 2. There are also programs in IMSL Math Library to solve linear equations subject to linear constraints.

Table 2 Mass distribution for the $D$-optimal design for second-order terms

4 Efficiencies

The D-efficiency of a design $\xi$ for estimating p parameters is defined as $\left[ {\left| {M\left( \xi \right)/}\right|M\left( {\xi^{D} } \right)|}\right]^{1/p}$ where $\xi^{D}$ is the D-optimal design. It is of interest to investigate the efficiencies of $\xi_{2.1}^{D}$in situations for which it is not optimal. Depending on the conclusions from the testing of hypothesis about $\theta_{2}$, the design $\xi_{2.1}^{D}$ has to be compared with different optimal designs.

4.1 D-Efficiencies of $\xi_{2.1}^{D}$ for First-Order Model

If after performing an experiment in accordance with $\xi_{2.1}^{D}$the null hypothesis $H_{0} :\theta_{2} =0$ is accepted and the experimenter does not have resources to carry out further trials then the experimenter has to remain satisfied with the data already collected. In this case, it is of interest to know how well $\xi_{2.1}^{D}$ performs in comparison with the D-optimal first-order design say $\xi_{1}^{D}$, which maximizes $\left| {M_{11} \left( \xi \right)} \right| =\left| {{\text{diag}}\left\{ {1,\alpha_{2} I_{k} } \right\}} \right|= \alpha_{2}^{k}$ and clearly has $\alpha_{2} =1.$ Now, for a second-order design $\xi$ the relevant information matrix for the parameters corresponding to the first order model is

$$ M_{1.2} \left( \xi \right) = M_{11} \left(\xi \right) - M_{12} \left( \xi \right)M_{22}^{ - 1} \left( \xi \right)M_{21} \left( \xi \right)$$

with determinant is

$$ \left| {M_{1.2}\left( \xi \right)} \right| = \alpha_{2}^{k} \left[ {\alpha_{2} +\left( {k - 1} \right)\alpha_{22} - k\alpha_{2}^{2} } \right]/\left[{\alpha_{2} + \left( {k - 1} \right)\alpha_{22} } \right].$$

Therefore, if $\xi_{2.1}^{D}$is used as a second-order design, the null hypothesis is accepted and the experimenter is not ready to re-estimate the parameters of the first-order model, then it can be seen that the D-efficiency of $\xi_{2.1}^{D}$in estimating the relevant parameters is given by

$$ D_{1.2} \left({\xi_{2.1}^{D} } \right) = \left\{ {\frac{{\alpha_{2}^{k} \left[{\alpha_{2} + \left( {k - 1} \right)\alpha_{22} - k\alpha_{2}^{2} }\right]}}{{\alpha_{2} + \left( {k - 1} \right)\alpha_{22} }}}\right\}^{{1/\left( {k + 1} \right)}}$$

With $\alpha_{2}$ and $\alpha_{22}$ as in Table 1. The actual numerical values after rounding up to four decimal points, are shown in Table 3 for $k = 2$up to $k =20.$

Table 3 $D$-efficiency of ${\xi }_{2.1}^{D}$ in estimating first-order model and keeping second-order terms

It is seen from Table 3 that as $k$ increases, the D-efficiency of $\xi_{2.1}^{D}$ increases but starting from a very low value. The initial increases are reasonably large, but the rate of increase slows down when $k$ becomes large. In fact, $D_{1.2} \left( {_{2D}^{*} }\right)$ converges to $1$ as $k$ goes to infinity but the convergence is extremely slow. For example, when $k =100$, the efficiency is $0.9100$ and for $k =1000$, it is $0.9860$.

On the other hand, if the experimenter is willing do some re-computation and use $\xi_{2.1}^{D}$as a first-order design, that is re-estimate only the parameters of a first-order model discarding all the earlier computations, then the D-efficiency of $\xi_{2.1}^{D}$is given by $D_{1} \left( {\xi_{2.1}^{D} } \right) =\alpha_{2}^{{k/\left( {k + 1}\right)}}$ where $\alpha_{2}$is as in Table 1. The actual numerical values after rounding up to four decimal points Table 4 for $k =2$ up to $k =20$.

Table 4 $D$-efficiency of $\xi_{2.1}^{D}$in estimating first-order model ignoring second-order terms

It can be seen in Table 4 that the efficiency starts from a fairly high value when k = 2. It keeps increasing rapidly as $k$ increases. Also, the efficiency converges to $1$ as $k$ goes to infinity. However, the convergence is not too fast and the rate of increase slows down as $k$ becomes large. For example, when $k=100$, the efficiency is $0.9901$ and for $k=1000$, the efficiency is $0.9990$.

5 D-Efficiencies of ${\xi }_{2.1}^{D}$ for Full Second-Order Model

If after performing the experiment according to ${\xi }_{2.1}^{D}$ the null hypothesis is rejected but the experimenter doesn’t have resources to perform further trials, then the experimenter has to be satisfied with estimates of the parameters he has already computed. In this case, it is of interest to see how well ${\xi }_{2.1}^{D}$ performs against ${\xi }_{2}^{D}$ in estimating all the parameters. In order to make a fair comparison, one should also investigate how well ${\xi }_{2}^{D}$ performs in estimating the parameters corresponding to the second-order terms in the model.

The moments of the optimal design ${\xi }_{2}^{D}$ for the full set of parameters also may be derived using the trick used in the Sect. 3.1 [3]. The results thus obtained are presented in Table 5 and clearly match the results available in the literature.

Table 5 Nonzero moments of $D$-optimal designs measures ${\xi }_{2}^{D}$

Moreover, interestingly the values of $t$ corresponding to $k$ in Table 1 are identical to the values of $t$ corresponding to $k+1$ in Table 5. This can be checked theoretically by comparing the two formulas for $t$.

In Table 5, it can be seen that as $k$ increases, the moments ${\alpha }_{2}(={\alpha }_{4})$ and ${\alpha }_{22}$ increase. Also, $\left| {M_{2} \left( {\xi_{2}^{D} } \right)}\right|$ is decreasing as $k$ increases.

Now, the full second-order model contains $p = \left( {\begin{array}{*{20}c} {k + 2} \\2 \\ \end{array} } \right) = \left( {k + 1} \right)\left( {k + 2}\right)/2$ parameters. Therefore, the D-efficiency of $\xi_{2.1}^{D}$ is given by

$$ \begin{aligned} D_{2} \left( {\xi _{{2.1}}^{D} } \right) & = \left\{ {\frac{{\left| {M\left( {\xi _{{2.1}}^{D} } \right)} \right|}}{{\left| {M\left( {\xi _{2}^{D} } \right)} \right|}}} \right\}^{{1/{\tiny\left( {\begin{array}{*{20}c} {k + 2} \\ 2 \\ \end{array} } \right)}}} \\ & = \left\{ {\frac{{\alpha _{2}^{k} \left( {\alpha _{2} - \alpha _{{22}} } \right)^{{k - 1}} \left[ {\alpha _{2} + \left( {k - 1} \right)\alpha _{{22}} - k\alpha _{2}^{2} } \right]\alpha _{{22}}^{{{\tiny\left( {\begin{array}{*{20}c} k \\ 2 \\ \end{array} } \right)}}} ~for\xi ~_{{2.1}}^{D} }}{{\alpha _{2}^{k} \left( {\alpha _{2} - \alpha _{{22}} } \right)^{{k - 1}} \left[ {\alpha _{2} + \left( {k - 1} \right)\alpha _{{22}} - k\alpha _{2}^{2} } \right]\alpha _{{22}}^{{{\tiny\left( {\begin{array}{*{20}c} k \\ 2 \\ \end{array} } \right)}}} ~for\xi ~_{2}^{D} }}} \right\}^{{1/{\tiny\left( {\begin{array}{*{20}c} {k + 2} \\ 2 \\ \end{array} } \right)}}} \end{aligned} $$

where the appropriate values of ${\alpha }_{2}$ and ${\alpha }_{22}$ from Tables 1 and 5 (or from the formulae for $t$ and $u$) are put into the expression.

The numerical values, after rounding up to four decimal points, of the $D$-efficiency of the design ${\upxi }_{2.1}^{D}$ are displayed in Table 6 for $k=2$ up to $k=20.$

Table 6 $D$-efficiency of ${\upxi }_{2.1}^{D}$ in estimating all the parameters of the second-order model

Figure 1 presents a plot of the above efficiencies against the value of $k$. It is noticeable in Table 6 and Fig. 1 that the efficiency starts with a high value, increases rapidly initially and then slowly as $k$ becomes large. The value converges to unity and the convergence is very fast.

Fig. 1

Efficiency ${D}_{2}\left({\xi }_{2.1}^{D}\right)$ against the value of $k$

5.1 D-Efficiency of ${\xi }_{2}^{D}$ in Estimating the Second-Order Parameters

It is of interest to find out how good ${\upxi }_{2}^{D},$ the $D$-optimal design for estimating all parameters of a second-order model, is in estimating the parameters of the second-order terms in the model for which the $D$-optimal design is ${\upxi }_{2.1}^{D}$. The $D$-efficiency of ${\upxi }_{2}^{D}$ in estimating ${\theta }_{2}$ (the vector of parameters corresponding to the second-order terms in the model) in comparison with ${\upxi }_{2.1}^{D}$ is

$$\begin{aligned}D_{{2.1}} \left( {\xi _{{2D}} } \right) &= \left\{ {\frac{{\left|{M_{{2.1}} \left( {\xi {\text{~}}_{2}^{D} } \right)}\right|}}{{\left| {M_{{2.1}} \left( {{\text{~}}\xi _{{2.1}}^{D} }\right)} \right|}}} \right\}^{{1/\frac{{k\left( {k + 1}\right)}}{2}}} \\ &= \left\{ {\frac{{\left( {\alpha _{2} -\alpha _{{22}} } \right)^{{k - 1}} \left[ {\alpha _{2} + \left( {k- 1} \right)\alpha _{{22}} - k\alpha _{2}^{2} } \right]\alpha_{{22}}^{{{\tiny\left( {\begin{array}{*{20}c} k \\ 2 \\ \end{array} } \right)}}} {\text{~~}}~{\text{for~~}}\xi _{2}^{D}}}{{\left( {\alpha _{2} - \alpha _{{22}} } \right)^{{k - 1}} \left[{\alpha _{2} + \left( {k - 1} \right)\alpha _{{22}} - k\alpha_{2}^{2} } \right]\alpha _{{22}}^{{{\tiny\left( {\begin{array}{*{20}c}k \\ 2 \\ \end{array} } \right)}}}{\text{~~~for}}~{\text{~}}\xi _{{2.1}}^{D} }}}\right\}^{{1/\frac{{k\left( {k + 1} \right)}}{2}}} .\end{aligned} \\$$

To compute the efficiency above we have to put the values of ${\alpha }_{2}\,\, \mathrm{and }\,\,{\alpha }_{22}$ from Table 5 in the numerator and from Table 1 in the denominator. The numerical values of the efficiencies after rounding up to four decimal points are shown in Table 6 for $k=2$ up to $k=20$.

Figure 2 provides a plot of the efficiencies of ${\upxi }_{2}^{D}$ against the value of $k$.

Fig. 2

Efficiency of ${\upxi }_{2}^{D}$ versus the value of $k$

It is seen in Table 7 and Fig. 2 that as $k$increases, efficiency of ${\upxi }_{2}^{D}$ increases and remains stable. The values converge to unity and the convergence is fairly rapid. However, it is noticeable that for all finite values of $k$, this efficiency is smaller than the efficiency of ${\upxi }_{2.1}^{D}$ in estimating all parameters of the model.

Table 7 $D$-Efficiency of ${\upxi }_{2}^{D}$ in estimating the second-order parameters

6 ${\varvec{D}}$-Optimal Product Designs for Second-Order Parameters

A $k$-dimensional design $\upxi$ over $\Omega$ is called a product design if it is of the form $\upxi = \left({\upxi }_{1}\times {\upxi }_{2}\times \dots \times {\upxi }_{k}\right)$ where ${\upxi }_{i}$ is a design on $\left[-1,1\right]$. It is symmetric if each ${\upxi }_{i}$ is so and it is permutation invariant if all the ${\upxi }_{i}$ are identical. Consider a symmetric, permutation invariant product design $\upxi$ supported at points with coordinates $0$ and ± 1 only. Let the $1$-dimensional design put a mass $1-w$ at $0$ and $\frac{w}{2}$ at each of ± 1. Then, for this design ${\upxi }_{*}$ we have ${\alpha }_{2}=w={\alpha }_{4}$. The symmetric permutation invariant product design $\upxi$ over $\Omega$, is generated from ${\upxi }_{*}$ and has ${\alpha }_{2}=w=$ $\alpha_{4}$ and $\alpha_{22} = w^{2}.$ Hence,

$$ \begin{aligned} &\alpha_{4} - \alpha_{22} = w - w^{2} = w\left( {1 - w}\right),\,{\text{and}} \\ & \alpha_{4} + \left( {k - 1}\right)\alpha_{22} - k\alpha_{2}^{2} = w + \left( {k - 1}\right)w^{2} - kw^{2} = w\left( {1 - w} \right). \\ \end{aligned}$$

Suppose among this class of designs ${\upxi }_{2.1}^{DP}$ maximizes $\left|{M}_{2.1}\left(\upxi \right)\right|$, i.e., within the class of symmetric product designs ${\upxi }_{2.1}^{DP}$ is $D$-optimal for the second-order parameters. Now for a symmetric product design $\upxi$,

$$ \left| {M_{2.1}\left( \xi \right)} \right| = \left( {\alpha_{4} - \alpha_{22} }\right)^{k - 1} \left[ {\alpha_{4} + \left( {k - 1}\right)\alpha_{22} - k\alpha_{2}^{2} } \right]\alpha_{22}^{{\left({\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} } \right)}} =w^{{k^{2} }} \left( {1 - w} \right)^{k} .$$

Clearly, $\left|{M}_{2.1}\left(\upxi \right)\right|$ is maximized with respect to $w$ when $w=k/(k+1)$. Therefore, the design ${\upxi }_{2.1}^{DP}$ is the symmetric permutation invariant product design obtained from the $1$-dimensional designs ${\upxi }_{*}$ putting a mass $k/\{2\left(k+1\right)\}$ at each of ± 1 and a mass $1/(k+1)$ at $0$.

The D-efficiency of ${\upxi }_{2.1}^{DP}$ for estimating parameters corresponding to the second-order terms of the model is given by

$$\begin{aligned} D_{{2.1}} \left( {\xi _{{2.1}}^{{DP}} } \right) &= \left\{{\frac{{\left| {M_{{2.1}} \xi _{{2.1}}^{{DP}} } \right|}}{{\left|{M_{{2.1}} \left( {\xi _{{2.1}}^{D} } \right)} \right|}}}\right\}^{{1/\frac{{k\left( {k + 1} \right)}}{2}}} \\ &= \left\{{\frac{{\left\{ {k/\left( {k + 1} \right)} \right\}^{{k^{2} }}\quad \left\{ {1/\left( {k + 1} \right)} \right\}^{k} ~~}}{{\left({\alpha _{2} - \alpha _{{22}} } \right)^{{k - 1}} \left[ {\alpha_{2} + \left( {k - 1} \right)\alpha _{{22}} - k\alpha _{2}^{2} }\right]\alpha _{{22}}^{{\left( {\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} } \right)}} \quad {\text{for}}~\xi_{{2D}}^{*}}}} \right\}^{{2/k\left( {k + 1} \right)}} ,\end{aligned} $$

where the values of $\alpha_{2}\,{\text{and}}\,\alpha_{22}$ in the denominator are put in from Table 1 (or the formulae derived). Numerical values of $D_{2.1} \left( {\xi_{2.1}^{DP} }\right)$ for $k =2$ up to $k =20$ are displayed Table 8.

Table 8 Nonzero moments of $D$-optimal product design ${\upxi }_{2.1}^{DP}$ and its efficiency

It can be seen from Table 8 that the moments ${\alpha }_{2}$ and ${\alpha }_{22}$ increase as $k$increases. Figure 3 provides a plot of the efficiency of ${\upxi }_{2.1}^{DP}$ against $k$.

Fig. 3

Efficiency ${D}_{2.1}\left({\upxi }_{2DP}^{*}\right)$ against $k$

Table 8 and Fig. 3 illustrate that the efficiency ${D}_{1.2}\left({\upxi }_{2DP}^{*}\right)$ increases and approaches $1$ as $k$goes to infinity. However, the convergence is not very fast.

7 ${\varvec{D}}$-Optimal Central Composite Designs (CCD) for Second-Order Parameters

In the design space $\Omega$, the set of points with coordinates 0 and 1 only has cardinality ${3}^{k}.$These ${3}^{k}$points may be partitioned into $k+1$sets with ${j}^{th}$set ${\psi }_{j}$ having all points with $j$ nonzero coordinates. The cardinality of the ${\psi }_{j}$ is $\left(\begin{array}{c}k\\j\end{array}\right){2}^{j}$$\left(j=0, 1, 2,\dots ,k\right)$. For actual experimentation, it is of interest to find designs that are supported on the sets with small number of points. One such design is the central composite design (CCD) which is defined as having nonzero masses ${w}_{j}$equally distributed over ${\psi }_{j}$ for $j=0,1,$ and $k$ only. Thus, the support points of the CCD are the origin, the vertices, and the middle points of the edges of the hypercube ${\left[-1,1\right]}^{k}$. One may alternatively say that a CCD having ${\alpha }_{4}={\alpha }_{2}$ puts a mass $1-s$ at the origin $\left(0, \dots ,0\right)$, and masses $sw$ and $s\left(1-w\right)$ equally distributed over the vertices $\left(\pm 1,\pm 1, \dots , \pm 1\right)$ and the axial points $\left\{\left(\pm \mathrm{1,0},\dots ,0\right),\left(0,\pm 1, 0,\dots , 0\right),\left(0,\dots , 0,\pm 1\right)\right\}$, respectively. For such a design,

$$ \alpha_{2} = s\left( {w + \left( {1 - w}\right)/k} \right),\quad \alpha_{22} = sw.$$

Among the CCD’s let ${\upxi }_{2.1}^{DC}$ be $D$-optimal for estimating parameters corresponding to the second-order terms, that is, let $\left|{M}_{2.1}\left(\upxi \right)\right|$ be maximized by ${\upxi }_{2.1}^{DC}$ among the CCD’s. For $k=2$, it is possible to find a CCD with the values of ${\alpha }_{2}$ and ${\alpha }_{22}$ matching those represented in Table 3. This is the CCD with $s=0.8242$ and $w=0.5732$. This CCD is thus $D$-optimal not only among the CCD’s but among all designs. On the other hand, for $k\ge 3$, the $D$-optimal CCD always has $s=1$ and hence it has

$$ \alpha_{2} = \left( {w + \left( {1 - w}\right)/k} \right),\alpha_{22} = w.$$

Substituting the above values reduces $\left|{M}_{2.1}\left(\xi \right)\right|$ to the following

$$ \left| {M_{2.1} \left(\xi \right)} \right|= \left( {\frac{1 - w}{k} } \right)^{k} w^{{\frac{{k\left( {k - 1}\right) + 2}}{2}}} \left( {k - 1} \right)^{2} .$$

It is readily seen that the value of $w$ maximizing $\left|{M}_{2.1}\left(\upxi \right)\right|$ is given by $w=({k}^{2}-k+2)/({k}^{2}+k+2).$

Thus, for $k>2$ the $D$-optimal CCD ${\upxi }_{2.1}^{DC}$ puts a mass $w=\left({k}^{2}-k+2\right)/\left({k}^{2}+k+2\right)$ distributed equally over the ${2}^{k}$vertices and a mass $1-w =2k/\left({k}^{2}+k+2\right)$ distributed equally over the $2k$ axial points. It is to be noted that instead of taking the entire set of vertices a suitable fraction (Resolution V fraction) is actually sufficient for estimation purposes.

It is of interest to investigate the efficiency of ${\upxi }_{2.1}^{DC}$ in estimating the parameters corresponding to the second-order terms of the model. The $D$-efficiency of ${\upxi }_{2.1}^{DC}$ relative to ${\upxi }_{2.1}^{D}$ is.

$$\begin{aligned} D_{{2.1}} \left( {\xi _{{2.1}}^{{DC}} }\right)& = \left\{ {\frac{{\left| {M_{{2.1}} \left( {\xi_{{2.1}}^{{DC}} } \right)} \right|}}{{\left| {M_{{2.1}} \left( {\xi_{{2.1}}^{D} } \right)} \right|}}} \right\}^{{1/\frac{{k\left( {k +1} \right)}}{2}}} \\ & = \left\{ {\frac{{\left( {\alpha _{2} - \alpha_{{22}} } \right)^{{k - 1}} \left[ {\alpha _{2} + \left( {k - 1}\right)\alpha _{{22}} - k\alpha _{2}^{2} } \right]\alpha_{{22}}^{{\left( {\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} } \right)}} {\text{~~}}\,{\text{for}}\,\xi{\text{~}}_{{2.1}}^{{DC}} }}{{\left( {\alpha _{2} - \alpha _{{22}}} \right)^{{k - 1}} \left[ {\alpha _{2} + \left( {k - 1}\right)\alpha _{{22}} - k\alpha _{2}^{2} } \right]\alpha_{{22}}^{{\left( {\begin{subarray}{*{20}c} k \\ 2 \\ \end{subarray} } \right)}} {\text{~~}}\,{\text{for}}\,~\xi_{{2.1}}^{D} }}} \right\}^{{1/\frac{{k\left( {k + 1}\right)}}{2}}}.\end{aligned}$$

Table 9 displays the values of $s,w,{\alpha }_{2}$, ${\alpha }_{22}$ the D-efficiency of ${\upxi }_{2.1}^{DC}$ after rounding up to four decimal points, for $k=2$ up to $k=20$.

Table 9 Nonzero moments and the D-efficiency of ${\upxi }_{2.1}^{DC}$

It can be seen in Table 9 that the moments ${\alpha }_{2}$ and ${\alpha }_{22}$ increase as $k$ increases. Figure 4 provides a plot of the D-efficiency ${D}_{2.1}\left({\upxi }_{2.1}^{DC}\right)$ of ${\upxi }_{2.1}^{DC}$ against $k$.

Fig. 4

Efficiency ${D}_{2.1}\left({\upxi }_{2.1}^{DC}\right)$ versus $k$

It is seen in Table 9 and Fig. 4 that the efficiency ${D}_{2.1}\left({\upxi }_{2.1}^{DC}\right)$ is decreasing from $k=2$ to $k=11$ but from $k=13$ it starts to increase. Further, ${D}_{2.1}\left({\upxi }_{2.1}^{DC}\right)$ converges to $1$ as $k$ goes to infinity but the increase is extremely slow. For example, when $k=50$, we computed the efficiency, and it is $0.9705$.

8 Comments and Discussion

This article considers experimental design for second-order polynomial regression in many variables over the hypercubic design space. Optimal designs under $D$-optimality criterion are derived for the situation when primary interest lies in estimating the parameters corresponding to the second-order terms in the model.

Optimal designs ${\upxi }_{2.1}^{D}$ among all designs as well as those optimal among the commonly used Central Composite Designs (CCD’s) and among the symmetric permutation invariant product designs are derived. The efficiencies of the optimal designs in these classes relative to the overall optimal designs are investigated. Also, the efficiencies of ${\upxi }_{2.1}^{D}$ in estimating the parameters of a first-order model are investigated. As expected, it is found that ${\upxi }_{2.1}^{D}$ is quite good in estimating a first-order model if the parameters corresponding to the second-order terms are ignored.

Comparisons of the performance of ${\upxi }_{2.1}^{D}$ and ${\upxi }_{2}^{D}$ gave some interesting results. Both were nearly equally good and had high efficiency in the situation where the other was optimal. However, for every k, the value of ${D}_{2}\left({\upxi }_{2.1}^{D}\right)$ was slightly larger than the corresponding value of ${D}_{2.1}\left({\upxi }_{2}^{D}\right)$. In view of this, it is recommended that if the experimenter is not really sure about necessity of the second-order terms in the assumed model then he should use ${\upxi }_{2.1}^{D}$.

It is to be noted that the designs under study are not exact (integer) designs and exact designs with moments matching the moments of the derived optimal designs may or may not exist. However, the exact designs obtained by multiplying the masses by the total number of trials N and rounding up to integers usually provides a highly efficient exact design which is often the best one can get for a given N.