Keywords

1 Introduction

In field sampling and social science research, creating samples that are representative of the population is important. This can be achieved by using stratified sampling, cluster sampling, or post-stratified sampling designs. In certain cases, the stratification variable may be subjective, rough, and imprecise, but can still provide valuable information about the relative position of a sample unit in a small set. Such stratification variable can be used to reduce the sampling variation, and cost in ranked set and judgment post-stratified sampling designs. These designs stratify the sample into groups of homogeneous observations using sample units’ relative positions (ranks) in small comparison sets.

For a ranked set sample (RSS) of size n, one first determines a set size H and then selects nH units at random from the population. These units are divided into n comparison sets, each of size H. Units in the comparison sets are then ranked from the smallest to the largest, without measurement. Ranking can be performed on either the variable of interest assessed on a less elaborate scale or an auxiliary variable. The unit judged to be the h-th smallest (Y [h]j) is measured in n h comparison sets for j = 1, …, n h, \(\sum _{h=1}^H n_h =n\). The measured observations Y [h]j, j = 1, …, n h;h = 1, …, H are called a ranked set sample. If n h = d for all h = 1, …, H so that n = dH, the RSS is called balanced, and d is called the cycle size. If there is no ranking error, the square brackets are replaced with round parentheses, and the Y (h)j becomes the h-th order statistic in a sample of size H.

Ranked set sampling design was introduced by McIntyre (1952, 2005). The main motivation in McIntyre’s work was to enable field researchers to conduct pasture yield (and similar) field assessments in an objective and efficient way. Takahasi and Wakimoto (1968) developed the theoretical foundation of the ranked set sampling design and showed that the RSS mean is always better than a sample mean of a simple random sample (SRS). Dell and Clutter (1972) showed that even with some ranking errors, the RSS mean is as good as, or better than, the SRS mean depending on the quality of ranking information. Research activities in RSS designs then expanded in different directions, including parametric and nonparametric settings. In the parametric setting, a few representative publications are Stokes (1995), Chen and Bai (2000), Arslan and Ozturk (2013), Hatefi et al. (2014), and Hatefi et al. (2015). In the nonparametric setting, readers are referred to Bohn and Wolfe (1992, 1994), Hettmansperger (1995), Koti and Babu (1996), Ozturk (1999), and Fligner and MacEachern (2006). Two books have been published on ranked set sampling design, Chen et al. (2003), and Bouza and Al-Omari (2019). A comprehensive list of references can be found in these publications.

The RSS research activities also considered the finite population setting. Patil et al. (1995) constructed an RSS using sampling without replacement selection procedure. Deshpande et al. (2006) expanded the RSS design to three different schemes of sampling without replacement. Frey (2011), Ozturk and Jafari Jozani (2014), and Jafari Jozani and Johnson (2011) used probability sampling and constructed Horvitz-Thompson-type estimators. Ozturk and Bayramoglu Kavlak (2018) constructed inference using a superpopulation model in ranked set sampling.

MacEachern et al. (2004) introduced the judgment post-stratification design to provide the flexibility for a multipurpose analysis of sample data. For a judgment post-stratified sample (JPS), one first selects and measures an SRS of size n, Y i, i = 1, …, n. For each measured unit Y i, one then selects additional H − 1 units from the population, without directly measuring them, to form a comparison set of size H. The units in the comparison set are ranked from the smallest to the largest, and the rank of Y i, R i is recorded. The pairs (Y i, R i), i = 1, …, n, constitute a JPS.

In recent years, the JPS design in an infinite population setting has generated extensive research interest. Ozturk (2014) considered the estimation of the population quantile and variance from a JPS. Wang et al. (2006) used the concomitant order statistics to estimate the population mean. Frey and Feeman (2012, 2013) constructed estimators for the population mean and variance by conditioning on the judgment group sample sizes. These new estimators improve the unconditional JPS estimators. Chen et al. (2014), Frey and Ozturk (2011), Wang et al. (2012), Wang et al. (2008), and Stokes et al. (2007) constructed constrained estimators using stochastic ordering among judgment ranking groups. The main idea in the constraint estimators is to minimize the impact of ranking error by forcing judgment class means to follow the stochastic order among ranking groups. In a different direction, Ozturk (2017) constructed conditional ranks in smaller comparison sets of size K < H given the original ranks in a larger comparison set of size H. The impact of any ranking error on the estimator in this case was relatively small, and less than for the estimator based on the large comparison set of size H. Ozturk (2013) and Ozturk and Demirel (2016) used a multi-ranking approach to reduce the impact of ranking error in judgment post-stratified and ranked set samples.

In the finite population settings, Ozturk (2016a, 2016b, 2019) constructed estimators for the population mean and total for the JPS design. A JPS can be constructed by sampling with or without replacement. It is shown that the variance estimator of the sample mean requires a finite population correction factor when sampling without replacement. Ozturk and Bayramoglu Kavlak (2018, 2019, 2020) developed inference to predict the population mean and total using a superpopulation model.

In the JPS design, the ranks are constructed post-experimentally after an SRS is chosen. Hence, it is possible to have more than one rank for each measured unit in the SRS by permuting the n(H − 1) unmeasured units used in the construction of comparison sets in the first created JPS. Each permutation creates n comparison sets, each of size H, containing the measured unit. The units in the sets are ranked again, without measurement, and the ranks of the measured units in the comparison sets are determined. This permutation procedure can be done many times and each permutation creates a new set of ranks for the same measured values. Ranks from different permutations are conditionally independent given the original SRS. One may then combine all these ranks using the Rao-Blackwell theorem by conditioning on the original SRS.

A similar idea can be used in the RSS design, but the extension to multiple ranks is not as trivial as in the JPS design. In the RSS design, the measured observations, Y [h]j, are not identically distributed. Hence, the units in the comparison set constructed after the permutation of n(H − 1) units are not iid since each comparison set contains one of the y [h]j from the original ranked set sample and this will have a different distribution from the other units in the set. Even though the comparison sets will be different after each permutation, the rank of y [h]j will depend on the original rank h. Hence, the idea of multiple ranks in the judgment post-stratified sampling may not be easily extended to ranked set sampling.

There are a few other differences between the RSS and JPS designs. One of the major differences is whether the ranking is done before or after the units are measured for the variable of interest. In RSS, the ranking is performed before one measures the units, and the ranks guide the measurement decision. The rank and the measurement of a unit cannot be separated. Hence, an RSS cannot be reduced to an SRS, unless it is unusual situation where the ranking variable is not correlated with the measurement variable. In a JPS, ranking is performed after one measures the units in the SRS. The ranks are not the essential part of the measured units; they are the ranks of the variable of interest measured on a quicker scale (e.g., visual inspection) after the construction of an SRS. Since the auxiliary (ranking) variable is only post-associated with the response measurements, it can be ignored and a JPS can be reduced to an SRS if desired.

Another major difference is the distributional properties of the ranks. The ranks in RSS are pre-determined nonrandom constants. Hence, the ranking group sizes n h, h = 1, …, H, are nonrandom integers. In a JPS, the rank R i is a discrete uniform random variable with the support on integers 1, …, H. Hence, the judgment group sample size vector (n 1, …, n H) has a multinomial distribution with the sample size n and the success probability vector (1 ∕H, …, 1 ∕H).

One may look at the RSS and JPS designs in terms of the trade-off between the efficiency gain of the RSS and the adaptability of JPS for multipurpose studies. To our knowledge, this trade-off has not yet been posed and investigated. In this paper, we provide a comprehensive study to compare the RSS and JPS designs for their efficiencies and multiple ranking properties. In Sect. 2, we provide a detailed description of multi-ranking in RSS and JPS designs. In Sect. 3, we review the distributional properties of the RSS and JPS means. In Sect. 4, we present empirical results to compare the RSS and JPS designs. In Sect. 5, we illustrate the use of RSS and JPS designs with an agricultural application example. Section 6 provides concluding remarks.

2 Sampling Designs with Multiple Ranking Methods

We consider a finite population of size N. The population values of the variable Y  are denoted as y 1, …, y n. The mean and variance of the population are given by

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{y}_N= \frac{1}{N}\sum_{i=1}^N y_i, \quad S_N^2=\frac{1}{N}\sum_{i=1}^N(y_i-\bar{y}_N)^2. \end{array} \end{aligned} $$

From this finite population, we construct RSS and JPS with multiple ranks. The samples are constructed using the sampling with and without replacement selection procedures. Unless stated otherwise, we always consider a finite population setting in this paper.

RSS with Multiple Ranks

We first consider an RSS selected using the sampling with replacement (SWR) selection procedure. For cycle j and rank h, we construct a comparison set of size H using a sampling without replacement (SWOR) scheme. The units in the comparison set are ranked by the best ranking method available. The unit judged to be the h-th smallest, Y [h]j, is measured. For the observation Y [h]j, additional K − 1 ranks can be constructed in two ways. If there are K − 1 (K > 1) rankers or ranking variables available, the rank of Y [h]j, R k|j,h, among the units in the comparison set is determined for each method k, k = 2, …, K. After these ranks are determined, all units in the comparison set are returned to the population before constructing the next comparison set. Hence, the same unit may appear in the final sample more than once, and all the observations are independent. We note that units within a comparison set are selected using the SWOR procedure to minimize the ranking error. The ranks using the first ranking method (k = 1) are predetermined (nonrandom constants) to have a balanced ranked set sample, n h = d, for h = 1, …, H. The remaining K − 1 ranks are random and may not necessarily be balanced.

Even if there is only one ranker or one auxiliary variable to rank the units, we can still construct an RSS with multiple ranks. For given values of h and j, Y [h]j is measured in a comparison set. Next, we form K − 1 different comparison sets by selecting H − 1 additional units at random from the population without measurement, \(V_{k|h,j}=\left \{Y_{[h]j},Y_{k,1},\ldots ,Y_{j,{H-1}}\right \}\), k = 2, …, K, and determine the rank of Y [h]j, R k|j,h, in each set for k = 2, …, K. The RSS with multiple ranks can be written as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \left\{ (Y_{[h]j},R_{k|j,h}), h=1,\ldots, H, j=1,\ldots, d; k=1,\ldots,K\right\}, \end{array} \end{aligned} $$

where R k|j,h is the conditional rank assigned by ranking method k given that the observation Y [h]j is assigned rank h. We note that P(R 1|j,h = h) = 1. The ranks assigned by another ranking method are random variables, but their distributions depend on the ranks assigned by the first (best) ranking method.

An RSS with multiple ranks using a SWOR selection scheme can be constructed in a similar fashion. The only difference is that after determining the rank of Y [h]j, all H units in the comparison set are removed from the population before constructing the next comparison set. Hence, for each ranking method, all comparison sets are disjoint.

The final sample cannot have repeated observations and the observations are not independent. If the population size N is large with respect to sample size n, ranked set samples constructed using SWR or SWOR selection procedures become approximately equivalent.

The construction of ranked set samples using multiple ranking methods is illustrated in Table 1. In this table, the third column presents the comparison sets in which a balanced ranked set sample is constructed with the first ranking method. It highlights that the units are ranked using ranking method 1; the bold-faced values are measured. The fourth column lists the ranks obtained from all K (K = 3) different ranking methods for the bold-faced values in column 3. The last column gives the ranked set sample of size 6. In this example, each entry has three ranks generated by three ranking methods.

Table 1 Illustration of RSS multi-ranker sampling with set size H = 3, cycle size d = 2, and the number of ranking methods K = 3
Full size table

JPS with Multiple Ranks

We first construct a simple random sample of size n using the SWR selection procedure and measure all n units, Y 1, …, Y n. For each Y i, we then select additional H − 1 units under SWOR selection from the population to form a comparison set V i = {Y i, Y 1, …, Y H−1}. We rank these units from smallest to largest without measuring Y , using K different ranking methods, and identify the rank of Y i, R k|i, for each ranking method k, k = 1, …, K, where R k|i is the rank of Y i assigned by ranking method k. All units in the comparison set, including the one we measured, are returned to the population before the construction of the next comparison set. Hence, a JPS may have repeated observations and all Y i, i = 1, …, n, are independent. This process creates the sample

$$\displaystyle \begin{aligned} \begin{array}{rcl} \{ Y_i,R_{k|i}\};\ i=1,\ldots, n, k=1,\ldots,K. \end{array} \end{aligned} $$

If only one ranking method is available, for each Y i, one can create K different comparison sets, V k|i = {Y i, Y 1,k, …, Y H−1,k} for k = 1, …, K, where Y h,k≠Y i is the additional unit selected from the population to construct the k-th comparison set. These sets are ranked using the ranking method and the ranks of Y i, R k|i, are determined in V k|i, for k = 1, …, K.

A JPS under the SWOR selection procedure is constructed in a similar fashion. The only difference here is that all comparison sets for each ranking method are disjoint, and hence, the JPS cannot have repeated observations. For small population sizes N, observations Y i, i = 1, …, n, in the sample are negatively correlated since sample units are selected as an SRS without replacement.

The construction of a JPS with multiple ranking methods and under the SWOR selection scheme is illustrated in Table 2. In this example, the sample and set sizes are 6 and 3, respectively. For each measured unit, three ranks are constructed (K = 3). The second column presents a simple random sample of size n = 6. The third column presents three comparison sets, V k|i, for each Y i, one for each ranking method. The fourth column presents the JPS with three ranks. The comparison sets of each ranking method in Table 2, sets in block 1, 2, or 3 in column 3, are disjoint and cannot have repeated observations. Comparison sets for the different ranking methods (sets in different blocks) are not necessarily disjoint because the same ranking unit can appear in more than one set in different ranking methods. Sampling is without replacement and thus the comparison sets in different rows for the same ranking method are disjoint. We note that the sample units will not be independent if the population size N is small in relation to the sample size n.

Table 2 Illustration of multi-ranker JPS under sampling without replacement selection with set size H = 3, sample size n = 6, and the number of ranking methods K = 3
Full size table

3 Statistical Inference Using RSS and JPS

In this section, we provide a brief overview of statistical inference using the RSS and JPS designs. We first assume K = 1. The estimators for the population mean are given as the sample mean of the RSS and JPS:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{Y}_{RSS}= \frac{1}{dH} \sum_{h=1}^H \sum_{j=1}^d Y_{[h]j} , \quad \bar{Y}_{JPS}= \frac{1}{d_n} \sum_{h=1}^H J_hI_h \sum_{j=1}^nY_{j}I(R_j=h), \end{array} \end{aligned} $$

where I(a) is 1 if a is true, I h = I(n h > 0), \(d_n=\sum _{h=1}^H I_h\), and J h = 1∕n h if n h > 0 and zero otherwise. Both of these estimators are unbiased for the population mean \(\bar {y}_N\) regardless of the ranking quality as long as a consistent ranking method is used. If all units in the comparison sets are ranked with the same ranking methods, the ranking procedure is called consistent. The following theorem provides variances of the sample means under SWR and SWOR selection schemes using a consistent ranking method.

Theorem 1

Let Y [h]j , h = 1, ⋯ , H, j = 1, …, d and (Y j, R j), j = 1, …, n be RSS and JPS constructed using a consistent ranking methods, respectively.

  1. (i)

    If the samples are constructed with replacement, the variances of \(\bar {Y}_{RSS}\) and \(\bar {Y}_{JPS}\) are given by

    $$\displaystyle \begin{aligned} \begin{array}{rcl} \sigma^2_{RSS}= \frac{1}{dH^2} \sum_{h=1}^H S^2_{[h]} \quad \sigma^2_{JPS}= \frac{H}{H-1} Var\left(\frac{I_{1}}{d_{n}}\right) \sum_{h=1}^H (\bar{y}_{[h]}- \bar{y}_N)^2 + E\left(\frac{I_{1}^2}{d_{n}^2n_{1}}\right) \sum_{h=1}^H S_{[h]}^2, \end{array} \end{aligned} $$

    where \(S^2_{[h]}= Var(Y_{[h]1})\), \(\bar {y}_{[h]}=E(Y_{[h]1})\), \(Var(I_1/d_n)=\frac {1}{H^2}\sum _{k=1}^{H-1}(\frac {k}{H})^{n-1}\) and

    $$\displaystyle \begin{aligned} \begin{array}{rcl} E\left(\frac{I_{1}^2}{n_{1}d_{n}^2}\right)=\frac{1}{H^{n}}\left(\frac{1}{n}+\sum_{k=2}^{H}\sum_{j=1}^{k-1}\sum_{t=1}^{n-k+1}\frac{(-1)^{j-1}}{k^2t}\binom{H-1}{k-1}\binom{k-1}{j-1}\binom{n}{t}(k-j)^{n-t}\right). \end{array} \end{aligned} $$
  2. (ii)

    If the samples are constructed without replacement, the variances of \(\bar {Y}_{RSS}\) and \(\bar {Y}_{JPS}\) are given by

    $$\displaystyle \begin{aligned} \begin{array}{rcl} \sigma^2_{RSS} & =&\displaystyle \frac{N-1-n}{n(N-1)} S^2_N- \frac{1}{nH} \sum_{h=1}^h \left( \bar{y}_{[h]}- \bar{y}_N\right)^2-\frac{1}{nH} \sum_{h=1}^H S_{[h,h]} \\ \sigma^2_{JPS}& =&\displaystyle C_1(n,H)\left\{ \sum_{h=1}^HS^2_{[h]}-\sum_{h=1}^H S_{[h,h]}\right\} + C_2(n,H,N)\frac{H^2S^2_N}{H-1}, {} \\ \end{array} \end{aligned} $$

    where S [h,h] = Cov(Y [h]1, Y [h]2),

    $$\displaystyle \begin{aligned} \begin{array}{rcl} C_1(n,H)& =&\displaystyle \left\{\frac{1}{H(H-1)}+E\left(\frac{I_{1}^2}{d_{n}^2n_{1}}\right)-\frac{H}{H-1}E\left(\frac{I_{1}^2}{d_{n}^2}\right)\right\} \\ C_2(n,H,N) & =&\displaystyle \left\{ {Var\left(\frac{I_{1}}{d_{n}}\right)}-\frac{1}{N-1}\left\{ \frac{1}{H}-E\left(\frac{I_{1}^2}{d_{n}^2}\right)\right\}\right\}. \end{array} \end{aligned} $$

The proofs of \(\sigma ^2_{JPS}\) in (i) and (ii) are given in Ozturk (2016a). The proof of \(\sigma ^2_{RSS}\) in (ii) is given in Patil et al. (1995). It is clear that the variance of the JPS mean involves expected values and variances of the functions of judgment group indicator function (I 1), sample sizes (n 1), and the number of non-empty judgment groups (d n). These quantities account for the variation due to the random sample sizes in judgment post-stratified samples. Ozturk (2016b) shows that as the sample size n becomes large, \(\sigma ^2_{JPS}\) approaches from above \(\sigma ^2_{RSS}\).

We now introduce unbiased estimators for \(\sigma ^2_{JPS}\) and \(\sigma ^2_{RSS}\). We first define the following quantities:

$$\displaystyle \begin{aligned} \begin{array}{rcl} U_1& =&\displaystyle \frac{1}{E\left(\frac{I_1I_2}{d_n^2}\right)}\sum_{h=1}^H\sum_{h\ne h'}^H \frac{I_hI_{h'}J_hJ_{h'}}{d_n^2}\sum_{i=1}^n\sum_{j=1}^n (Y_i-Y_j)^2 I(R_i=h)I(R_j=h'), \\ U_2& =&\displaystyle \sum_{h=1}^H\frac{HI_h^*J_hJ^*_h}{d_n^*} \sum_{i=1}^n\sum_{j\ne i}^n (Y_i-Y_j)^2 I(R_i=h)I(R_j=h), \\ U_1^*& =&\displaystyle \frac{1}{2d^2H^2} \sum_{h=1}^H\sum_{h'\ne h}^H \sum_{i=1}^d \sum_{j=1}^d (Y_{[h]i}-Y_{[h']j})^2 \\ U_2^*& =&\displaystyle \frac{1}{2d(d-1)H^2} \sum_{h=1}^H \sum_{i=1}^d \sum_{j\ne i}^d (Y_{[h]i}-Y_{[h]j})^2 , \end{array} \end{aligned} $$

where \(d_n^*= \sum _{h=1}^H I(n_h>1)\), and \(J^*_h= 1/(n_h-1)\) if n h > 1 and zero otherwise.

Theorem 2

Let Y [h]j , h = 1, ⋯ , H, j = 1, …, d and (Y j, R j), j = 1, …, n be RSS and JPS constructed using a consistent ranking method, respectively.

  1. (i)

    If the samples are constructed with replacement, d > 1 and at least one judgment group in a JPS has at least two observations, the unbiased variance estimators for \(\bar {Y}_{RSS}\) and \(\bar {Y}_{JPS}\) are given by

    $$\displaystyle \begin{aligned} \begin{array}{rcl} \hat{\sigma}^2_{JPS} & =&\displaystyle \frac{Var\left(I_{1}/d_{n}\right)}{2(H-1)}U_{1}+ \left\{E\left(\frac{I_{1}^2}{d_{n}^2n_{1}}\right)-Var\left(\frac{I_{1}}{d_{n}}\right)\right\} \frac{U_{2}}{2} \\ \hat{\sigma}_{RSS}^2 & =&\displaystyle \frac{U^*_2}{d}. \end{array} \end{aligned} $$
  2. (ii)

    If the samples are constructed without replacement, d > 1 and at least one judgment group in a JPS has at least two observations, the unbiased variance estimators for \(\hat {Y}_{RSS}\) and \(\bar {Y}_{JPS}\) are given by

    $$\displaystyle \begin{aligned} \begin{array}{rcl} \hat{\sigma}^2_{JPS} & =&\displaystyle C_1(n,H)U_{2}/2+C_2(n,H,N)\frac{(N-1)(U_{1}+U_{2,2})}{2N(H-1)}, {} \\ \hat{\sigma}_{RSS}^2 & =&\displaystyle \frac{U^*_2}{d}-\frac{U^*_1+U^*_2}{N}. \end{array} \end{aligned} $$

Theorem 2 provides unbiased estimators for the variance of the RSS and JPS means for an arbitrary but consistent ranking scheme when K = 1. An approximate (1 − α)100% confidence interval for the population mean can be constructed using the normal approximation:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{Y}_{RSS} & \pm&\displaystyle t_{1-\alpha/2, n-H} \hat{\sigma}_{RSS} \\ \bar{Y}_{JPS} & \pm&\displaystyle t_{1-\alpha/2, n-H} \hat{\sigma}_{JPS}, \end{array} \end{aligned} $$

where t a,df is the a-th upper quantile of the t-distribution with degrees of freedom df. The degrees of freedom df = n − H is suggested to account for the heterogeneity among ranking groups.

There are different ways to combine the ranking information in multi-ranking RSS and JPS designs. Ozturk and Kravchuk (2021a, 2021b) provided detailed developments of these procedures. In this paper, we only consider one of the approaches, in which each observation is weighted based on the agreement scores of the K ranking methods. Let \(w_{h',i}\) be the proportion of K ranking methods which assign rank h′ to the i-th observation in the sample:

$$\displaystyle \begin{aligned} \begin{array}{rcl} w_{h'|i,h} =\frac{1}{K} \sum_{k=1}^K I(R_{k|i,h}=h')/K,\ h'=1,\ldots, H , \mbox{ for the RSS} \end{array} \end{aligned} $$

and

$$\displaystyle \begin{aligned} \begin{array}{rcl} w_{h'|i} = \frac{1}{K} \sum_{k=1}^K I(R_{k|i}=h')/K, \ h'=1,\ldots, H, \mbox{ for the JPS}. \end{array} \end{aligned} $$

We estimate the population mean by allocating each observation into ranking group h′ based on how strong the agreement is among the K ranking methods to assign the observation to judgment group h′:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{Y}_{RSS,w} & =&\displaystyle \sum_{h'=1}^H \frac{J_{w,h'}}{d_w} \sum_{h=1}^H \sum_{i=1}^d Y_{[h]i} w_{h'|i,h}, \enspace J_{w,h'}=\left\{ \begin{array}{ll} \frac{1}{n_{w,h'}} &\displaystyle \mbox{ if } n_{w,h'} >0 \\ 0 & \mbox{otherwise} \end{array} \right. , \ n_{w,h'} =\sum_{i=1}^d \sum_{h=1}^H w_{h'|i,h}. \\ \bar{Y}_{JPS,w}& =&\displaystyle \sum_{h'=1}^H \frac{J_{w,h'}}{d_w} \sum_{h=1}^H \sum_{i=1}^d Y_{i} w_{h'|i}, \ J_{w,h'}=\left\{ \begin{array}{ll} \frac{1}{n_{w,h'}} &\displaystyle \mbox{ if } n_{w,h'} >0 \\ 0 & \mbox{otherwise} \end{array} \right. , \quad n_{w,h'} =\sum_{i=1}^d \sum_{h=1}^H w_{h'|i}, \end{array} \end{aligned} $$

where \(d_w =\sum _{h'=1}^H I(n_{w,h'} >0)\). In the expressions above, \(n_{w,h'}\) can be considered as the effective sample size for judgment group h′. The asymptotic distribution of \(\bar {Y}_{JPS,w}\) is considered in MacEachern et al. (2004) and Ozturk and Kravchuk (2021a). The asymptotic distribution of \(\bar {Y}_{RSS,w}\) is given in Ozturk and Kravchuk (2021b).

In this paper, we only consider the jackknife variance estimates of these estimators. Let \(\bar {Y}_{RSS,w}^{(-[h]i)}\) ( \(\bar {Y}_{JPS,w}^{(-i)}\)) be the RSS (JPS) estimator after the observation Y [h]i (Y i) and all ranks associated with it are removed from the sample. The jackknife variance estimates are given by

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat{\sigma}^2_{RSS,J}& =&\displaystyle {fpc}\frac{(n-1)^2}{n^2}\sum_{h=1}\sum_{i=1} \left( \bar{Y}_{RSS,w}^{(-[h]i)}- \bar{Y}_{RSS,w}^{-([.].)}\right)^2\\ \hat{\sigma}^2_{JPS,J} & =&\displaystyle fpc\frac{(n-1)^2}{n^2} \sum_{i=1}^n( \bar{Y} ^{(-i)}_{JPS,w}, -\bar{Y} ^{(.)}_{JPS,w})^2\\ fpc& =&\displaystyle \left\{\begin{array}{ll} 1-\frac{n}{N-1} &\displaystyle \mbox{ SWOR selection}\\ 1 & \mbox{ SWR selection.} \end{array} \right. \end{array} \end{aligned} $$

where fpc is the finite population correction factor, \(\bar {Y}_{RSS,w}^{-([.].)}= \frac {1}{dH} \sum _{h=1}^H \sum _{i=1}^d\) \(\bar {Y}_{RSS,w}^{-([h]i)}\) and \(\bar {Y} ^{(.)}_{JPS,w}=\frac {1}{n} \sum _{i=1}^n \bar {Y} ^{(-i)}_{JPS,w}\). In the jackknife variance estimates, we used the coefficient (n − 1)2∕n 2 since this coefficient provides smaller bias than the usual coefficient (n − 1)∕n, (Ozturk and Kravchuk, 2021a,b).

An approximate (1 − α)100% confidence interval for multi-ranking RSS and JPS designs can be constructed using the jackknife variance estimates:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \bar{Y}_{RSS,w} & \pm&\displaystyle t_{1-\alpha/2, n-H} \hat{\sigma}_{RSS,J} \\ \bar{Y}_{JPS,w} & \pm&\displaystyle t_{1-\alpha/2, n-H} \hat{\sigma}_{JPS,J}. \end{array} \end{aligned} $$

In the next section, we compare the RSS and JPS estimators in terms of their efficiencies and coverage probabilities for a varying degree of ranking quality and different set sizes.

4 Comparison of RSS and JPS Designs

We performed a simulation study to investigate the contrasting features of RSS and JPS estimators. In the simulation study, samples were generated from two finite populations with large population size N = n + 1000 and small population sizes N = nH + 50. We considered a normal, N(μ = 50, σ = 5), and a lognormal, LN(μ = 0, σ = 1), distribution. The population values of the response variable Y  were generated using the quantile functions:

$$\displaystyle \begin{aligned} \begin{array}{rcl} y_i=F^{-1}_N(i/(N+1), \mu=50, \sigma=5),\mbox{and } y_i=F^{-1}_{LN}(i/(N+1), \mu=0, \sigma=1), i =1 ,\ldots, N, \end{array} \end{aligned} $$

where \(F^{-1}_N(y, \mu , \sigma )\) and \(F^{-1}_{LN}(y, \mu , \sigma )\) are the inverse cumulative distribution functions of a normal distribution with location parameter μ and scale parameter σ and lognormal distribution with scale parameter exp(μ) and shape parameter σ, respectively. The samples were generated using SWR and SWOR selection procedures for both population sizes N = n + 1000 and N = nH + 50. The quality of ranking was modeled using a ranking variable X, such that X = Y + τ𝜖, where 𝜖 has a normal distribution with mean zero and variance 1 and independent of Y . The correlation coefficient between X and Y  is given by \(\rho =\frac {1}{\sqrt {1+\tau ^2/\sigma ^2}}\). The values of ρ were selected to be 0.01, 0.25, 0.5, 0.75, 0.9, 1 where values less than 1 will result in imperfect ranking. For the normal distribution, we fixed the sample size at n = 36 and varied the set sizes as H = 2, 3, 4, 6, 12 to explore the impact of different set sizes on the RSS and JPS designs. We purposely selected a smaller sample size n = 36 to evaluate the approximation of the coverage probabilities of the confidence intervals to the nominal coverage probability 0.95. The simulation size is taken to be 5000. An R-package RankedSetSampling (Ozturk et al., 2021) is used to compute the estimators and construct confidence intervals. The package is available to download at https://biometryhub.github.io/RankedSetSampling.

We first investigate the efficiencies of the RSS and JPS estimators. The relative efficiencies are defined as the ratio of the mean square errors of the RSS and JPS estimators:

$$\displaystyle \begin{aligned} \begin{array}{rcl} RE=\frac{MSE(JPS )}{MSE(RSS)}. \end{array} \end{aligned} $$

A value of RE greater than 1 indicates that the RSS estimator is more efficient than the JPS estimator. Figure 1 presents the relative efficiencies for the population size N = n + 1000 when samples were generated using the SWR selection procedure. The set sizes and the number of ranking methods are indicated in the legend on each panel. The first panel shows the relative efficiency curves when both RSS and JPS were generated with just one ranking method K = 1. It is clear in this case that the RSS estimator is more efficient. The efficiency gain is minimal for H = 2, 3, moderate for H = 4, 6, and substantial for H = 12. This intuitively makes sense since large set sizes lead to many judgment groups having no measured observations in a JPS. Empty ranking groups inflate the variance of the JPS estimator. The RE values are similar to each other for all ρ values when H = 2, 3, 4, 6, except for ρ when H = 12 where it increases.

Fig. 1
figure 1

Efficiency comparison of RSS and JPS designs under SWR selection for large-sized normal distribution population

Full size image

Figure 1 also presents the relative efficiencies in three different panels when different number of ranking methods (K=2, 5,10) is used. Comparing these panels with panel 1, one can see that the gain in RE values decreases with the number of ranking methods K. For example, the RE values in panel 1 (K = 1) are around 1.4 when ρ < 0.75, and it reduces essentially to 1 in panel 4 (K=10). Similar observation can be made in panel 2 (K = 2) and panel 3 (K = 5). Under perfect ranking, RSS is still superior to the JPS for all set sizes.

Figure 2 presents the efficiency curves for the small population size, N = nH + 50. In this part of the simulation, both the RSS and JPS were generated under the SWOR selection procedures. The efficiency results are similar to those in Fig. 1, with the key difference being that the RE curves are higher (lower) in Fig. 2 than in Fig. 1 when K = 1 and K = 2 (K = 5 and K = 10).

Fig. 2
figure 2

Efficiency comparison of RSS and JPS designs under SWOR selection for the small-sized normal distribution population

Full size image

These efficiency results indicate that RSS estimator is more efficient than the JPS estimator when the number of ranking methods is small (K = 1, 2). For the larger number of ranking methods (K = 5, 10), difference between the efficiency gain of RSS and JPS estimators diminishes.

We also investigated the coverage probabilities of the confidence intervals for the population mean. Figure 3 presents the coverage probabilities for the samples constructed with replacement from the population of size N = n + 1000. We note that confidence intervals are constructed using unbiased variance estimates when K = 1. For K≠1, we used the jackknife variance estimates. The panels in the first and second columns of Fig. 3 present the coverage probabilities of RSS and JPS confidence intervals for K = 1, 2, 5, 10, respectively. The coverage probabilities of RSS confidence intervals can be seen to be reasonably close to the nominal coverage probability of 0.95 when ρ ≤ 0.75 and K = 2, 5, 10, but they are slightly larger when ρ > 0.75 and K = 2, 5, 10. The coverage probabilities in the second column of Fig. 3 are reasonably close to the nominal coverage probability 0.95 for all ρ and K values in the simulation study.

Fig. 3
figure 3

Coverage probabilities of the jackknife confidence intervals under SWR selection for normal distribution

Full size image

Figure 4 presents the coverage probabilities for the population size N = nH + 50. In this case, coverage probabilities are again close to nominal coverage probability of 0.95 under imperfect ranking (ρ ≤ 0.75) for both RSS and JPS and K = 1, 2, 5, 10. Unlike Fig. 3, coverage probabilities are slightly inflated for both RSS and JPS confidence intervals when ρ > 0.75 and K = 2, 5, 10. Under perfect ranking (ρ = 1), jackknife variance estimator overestimates the variances of the RSS and JPS estimators and leads to a larger coverage probability than the nominal coverage probability of 0.95

Fig. 4
figure 4

Coverage probabilities of the jackknife confidence intervals under SWOR selection for normal distribution

Full size image

In the second part of the simulation study, we generated samples from the lognormal distribution with the scale parameter exp(μ)(μ = 0) and the shape parameter σ = 1. The sample and set sizes were as previously n = 48 and H = 2, 3, 4, 6, 12. All the other simulation parameters remained the same. The lognormal distribution is strongly positively skewed. For this reason, we increased the sample size from 36 to 48. Figures 5 and 6 present the relative efficiencies of the RSS and JPS estimators for large and small population sizes, respectively. The pattern of the efficiency curves is very similar to that for the normal population. The main difference is in the magnitude of the efficiency gain. The efficiency curves reach to higher values for the normal distribution. This result is consistent with the efficiency results of ranked set samples in (McIntyre, 1952, 2005). McIntyre reported that the efficiencies are higher for symmetric distributions (highest for the uniform distribution) and decrease with skewness. Since the lognormal distribution has strong skewness, the efficiencies are slightly lower than for the normal distribution.

Fig. 5
figure 5

Efficiency comparison of RSS and JPS designs under SWR selection for the large-sized lognormal distribution population

Full size image
Fig. 6
figure 6

Efficiency comparison of RSS and JPS designs under SWOR selection for the small-sized lognormal distribution population

Full size image

Figures 7 and 8 present the coverage probabilities of the jackknife confidence intervals of the population mean for the SWR and SWOR designs, respectively. It is clear that the coverage probabilities for the lognormal distribution are lower than the nominal coverage probability 0.95. The SWOR selection provides a better coverage probability than the SWR selection. Since a jackknife confidence interval relies on the normal approximation, the sample size n = 48 is not large enough for a good approximation when the underlying population is strongly skewed.

Fig. 7
figure 7

Coverage probabilities of the jackknife confidence intervals under SWR selection for large-sized lognormal distribution population

Full size image
Fig. 8
figure 8

Coverage probabilities of the jackknife confidence intervals under SWOR selection for small-sized lognormal distribution population

Full size image

5 Application

In this section, we use a real-life finite population example to compare the JPS and RSS estimators. The population consisted of 350 grapevine plants at Coombe vineyard at the University of Adelaide, Waite campus, Australia. The vineyard is used as a research and teaching facility. There are eight different rootstocks originally planted, on which Shiraz is grafted. These rootstocks are popular commercial choices in South Australia. The standard vineyard management of this population requires the monitoring and measuring of certain characteristics of vine plants. In this paper, we consider seven characteristics; X 1, trunk circumference (cm) in 2018; X 2, trunk circumference (cm) in 2019; X 3, shoot counts; X 4, total shoots; X 5, pruning weight (kg); X 6, cordon length (cm); and X 7, total bunch numbers and Y , nett fruit weight in 2019 (kg). Our interest was in the estimation of the mean nett fruit yield of this population of grapevines in 2019. The variables X i, i = 1, …, 7, were used as ranking variables in comparison sets, and hence, the number of ranking methods is K = 7. There were missing values on some vines, and after removing the plants having missing observations, the population size was reduced to N = 309. In this population, the correlation coefficients between Y  and X i, ρ i = cor(Y, X i) are ρ 1 = 0.240, ρ 2 = 0.191, ρ 3 = 0.310, ρ 4 = 0.321, ρ 5 = 0.172, ρ 6 = 0.274, and ρ 7 = 0.713. The mean and standard deviation of the Y  variable are 10.558 kg and 3.855 kg, respectively.

We performed another simulation study using these 309 vine plants. In each replication of the simulation study, we generated the single-ranking judgment post-stratified and ranked set samples with the ranking variable X 7 (K = 1), the multi-ranking judgment post-stratified and ranked set samples with X k, k = 1, …, 7 (K = 7), and a simple random sample. The sample sizes were selected to be n = 30 and 48. For the sample size n = 30, the set sizes were chosen H = 3, 5, 6, 10. For the sample sizes n = 48, the set sizes were H = 3, 4, 6. Samples were generated using the SWR and SWOR selection procedures. The simulation size was 5000.

Table 3 presents the relative efficiency of the multi-ranking RSS estimator (K = 7) with respect to the other four estimators: the JPS estimator with K = 7 and K = 1, the SRS estimator, and the RSS estimator with K = 1. When the entries in Table 3 are greater than one, the multi-ranking RSS estimator with K = 7 was superior. The other efficiency results can be obtained by taking the ratio of any two efficiency columns in Table 3. For example, the efficiency of the JPS estimator with K = 1 relative to the SRS estimator can be obtained by taking the ratio of column 6 and column 5. When n = 30, H = 3, and the replacement is true, this efficiency is calculated 1.246(1.321∕1.060 = 1.246). The other relative efficiencies can be computed in a similar fashion.

Table 3 Relative efficiency of the weighted RSS estimators with K = 7 for mean fruit yield of vine plants in Coombe vineyard. Entries greater than one indicate that the RSS estimator with K = 7 is more efficient
Full size table

All entries in Table 3 are greater than one which indicates that the RSS multi-ranking estimator with K = 7 is more efficient than JPS and SRS estimators. The efficiencies of RSS estimator with K = 7 with respect to JPS and SRS estimators increase with set sizes, but remain relatively constant with RSS estimator wit K = 1 (column 7). The reason for this is that the correlation coefficient between ranking variable X 7 and response is 0.729, while the other correlation coefficients are all less than 0.321. Hence, the improvement of ranking quality due to ranking variables with low correlation coefficients is minimal, and the relative efficiency for multi-ranker estimator remains relatively constant. For this particular population and ranking methods, the JPS estimators are more efficient than the SRS estimator and less efficient than multi-ranker RSS estimator.

We also computed the coverage probabilities of the confidence intervals based on the judgment post-stratified, simple random, and ranked set samples for the population mean. All coverage probabilities were reasonably close to the nominal coverage probability of 0.95. Due to space considerations, these empirical coverage probabilities are not reported here.

6 Concluding Remarks

Field research is expensive and time-consuming, particularly in natural environments where variables are difficult to control. If auxiliary variables are available, they can be used to account in the analysis for the inherent variation among sampling units. These auxiliary variables can be used as blocking variables if they can be evaluated in an objective manner. In certain settings, auxiliary variables may not be assessed accurately. Their assessment may be rough, imprecise, and subjective, but still helpful for ordering the units in a small set independently of knowing the actual values of the variable of interest.

Ranked set and judgment post-stratified sampling designs use this ordering information to construct samples that are more likely to span the full range of values in the population. It has been established in the literature that a ranked set sample is generally more efficient than a judgment post-stratified sample. However, RSS designs induce a strong structure in the sample. Hence, an RSS cannot be analyzed with the inferential procedures developed for an SRS design.

The JPS design may be less efficient than the RSS design, but the sample constructed can be reduced to a simple random sample, allowing the flexibility to perform multiple analyses of various responses on the same data set. This becomes useful if the data set is needed for a multipurpose study. In this paper, we show how to reduce the efficiency loss of a JPS with respect to an RSS by constructing multiple ranks for the response variable on each measured unit. Hence, the JPS design provides the flexibility for multipurpose analysis at the expense of little efficiency loss with respect to a balanced ranked set sample. Another advantage of the JPS design is that it is relatively straightforward to construct a multi-ranking JPS even when there are no additional auxiliary ranking variables, and this can be done by permuting the units selected to form comparison sets. This idea is not easily extended to a ranked set sampling. We would recommend that the JPS design should be considered in field sampling, especially for multipurpose studies.