Keywords

1 Introduction

Wildlife biologists and animal ecologists realized early on that their methods for surveying animal populations did not provide accurate counts. Animals are missed by virtually all survey methods, and biologists were forced to develop methods that produced not only counts of animals detected but also estimates of those present, but not detected. Several clever approaches have been developed to deal with this issue of nondetection (Seber, 1982; Williams et al., 2002; Seber & Schofield, 2019), and they have seen wide use in ecology and wildlife biology for decades. Another problem faced when surveying animal populations and communities is misclassification. Some of the same factors that cause animals to be difficult to detect (e.g., reliance on partial sightings or auditory cues) can also result in misclassification. For example, an auditory bird survey may result in species misidentification. A visual survey of a sexually monomorphic bird species may result in sex being misclassified. A sighting survey of reproduction in manatees may result in a reproductive female being misclassified as nonreproductive because her young is too far away from her or obscured by her body. Methods for dealing with nondetection have thus been modified to incorporate misclassification as well (Pradel, 2005; MacKenzie et al., 2018).

These same problems in counting and classifying characterize other scientific disciplines as well, but with much less corresponding effort to deal with them. Here, I first provide a brief history of both capture-recapture and closely related occupancy modeling. Then, I survey uses of these models in disciplines other than ecology and wildlife biology and identify opportunities for even greater use.

2 Capture-Recapture

In this section, I provide a brief introduction to capture-recapture modeling by describing the basic ideas underlying several classes of capture-recapture models. Data are typically summarized as capture histories depicting whether or not an individual was captured or detected at each sampling occasion of the study. Rather than developing the full likelihoods for these different classes of model, I define parameters and then write out probability structures for sample capture histories as an abbreviated way of explaining the thinking that underlies these models.

2.1 2-Sample, Closed Population, Single State

The most basic capture-recapture estimator is based on the recognition that the proportion of a specific type of individual or entity in a representative sample from a population should be roughly equal to that in the population itself. Let M be the known number of animals of a certain type in a total population of N individuals. Then define m as the number of animals of that type in a sample of n individuals. If the sample is representative of the population, in the sense of a similar proportion of marked animals, then we expect:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \frac{m}{n} \approx \frac{M}{N}. \end{array} \end{aligned} $$
(1)

We can rearrange (1) to obtain the following estimator for N:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} {\hat N} = \frac{n M}{m}. \end{array} \end{aligned} $$
(2)

Expression (2) is known as the Lincoln-Petersen estimator, and the prototypical study to which it applies entails two sampling occasions separated by a short time interval over which the sampled population is assumed to be “closed,” with no animals entering or departing. On occasion 1, M animals are captured and marks are applied to them. On occasion 2, another sample of n animals is captured, m of which are found to be marked. This estimator has been independently derived a number of times, initially by Laplace (1786), who used it to estimate the human population of France, and later by Lincoln (1930) to estimate the number of waterfowl in late summer to autumn in North America.

The estimator in (2) can also be viewed as a precursor to the general Horvitz and Thompson (1952) estimator for a population total. M is the number of animals sampled on occasion 1, and m∕n estimates p 1, the probability that a member of the population of size N is caught in occasion 1:

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} {\hat N} = \frac{M}{{\hat p}_{1}}. \end{array} \end{aligned} $$
(3)

The estimator of (2) and (3) and its associated variance have been derived using both hypergeometric (fixed sample size) and binomial (random sampling) likelihoods (reviewed by Seber, 1982, Williams et al., 2002, Seber & Schofield, 2019).

The statistics used in capture-recapture modeling are most frequently written as the number of animals exhibiting each possible capture or detection history. For a 2-sample study, there are only three such statistics denoted as x ij, where i = 1 if caught on occasion 1 and 0 if not caught then, and j = 1 if caught on occasion 2, and 0 if not.

  • x 11 = m =  number of animals caught on occasions 1 and 2,

  • x 10 = M − m =  number of animals caught on occasion 1 but not on occasion 2,

  • x 01 = n − m =  number of animals caught on occasion 2 but not on occasion 1.

The number of animals not captured at either sampling occasion, x 00, is unknown, and the problem of estimating total abundance, N, is equivalent to the problem of estimating x 00.

Given that an animal is a member of the sampled population, the probabilities of it exhibiting each observable capture history are then:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr(11) & = &\displaystyle p_{1} p_{2}, \\ \Pr(10) & = &\displaystyle p_{1} (1 - p_{2}), \\ \Pr(01) & = &\displaystyle (1 - p_{1}) p_{2}, \end{array} \end{aligned} $$

where p t is the capture probability for sampling occasion t. Under this 2-sample model, N, p 1, and p 2 can all be estimated. It is termed a “single state” model because all animals are assumed to have the same probabilities of appearing in a sample (i.e., no stratification by age, sex, size, etc.)

2.2 >2-Sample, Closed Population, Single State

This approach was extended to multiple sampling occasions (Schnabel, 1938; Darroch, 1958) for closed populations. Capture histories were modeled as in the 2-sample case. For a study with five sampling occasions, the probability of an animal in the sampled population showing a capture history of 01010 is:

$$\displaystyle \begin{aligned}\Pr(01010) = (1 - p_{1}) p_{2} (1- p_{3}) p_{4} (1- p_{5}).\end{aligned}$$

Each possible capture history has an associated probability such as above, and we know how many animals exhibited the history, so we can develop a corresponding likelihood and estimate the capture probabilities and abundance. Subsequent developments included consideration of behavioral response of animals to initial capture, heterogenous capture probabilities, and other generalizations (Otis et al., 1978; Chao and Huggins, 2005a 2005b; Seber & Schofield, 2019).

2.3 >2 Samples, Open Populations, Single State

Capture-recapture methods were extended to “open” populations as well, where sampling occasions could be separated by long time intervals such that gains and losses to the population could occur between occasions (e.g., Jackson, 1933, 1939; Cormack, 1964; Jolly, 1965; Seber, 1965). These models were also based on multinomial likelihoods and required additional parameters for survival of an animal from one sampling occasion to the next. For example, let p t denote capture probability for sampling occasion t, and let ϕ t denote the probability that an animal alive at sampling occasion t survives until occasion t + 1 and remains in the sampled population. The conditional probability associated with capture history x 01010 in a 5-occasion study is:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \Pr(01010| \mbox{release in 2}) = \phi_{2} (1 - p_{3}) \phi_{3} p_{4} (1 - \phi_{4} p_{5}). \end{array} \end{aligned} $$
(4)

The last terms in parentheses include both the possibility that the animal survived until 5 but was not caught and the possibility that the animal did not survive.

Likelihoods conditional on new releases in each sampling period can be used to estimate capture probabilities and survival probabilities and, assuming that animals that were and were not previously captured exhibit the same capture probabilities, occasion-specific abundance (see expression 3). Subsequent parameterizations include different ways of modeling the entry of new animals into the sampled population (e.g., Crosbie & Manly, 1981; Pradel, 1996; Schwarz & Arnason, 1996), as opposed to simply conditioning on entries as in (4).

2.4 >2 Samples, Open Populations, Multiple States

Arnason (1972, 1973) introduced the concept of multiple states in which an animal could be captured, where states initially represented different locations and were later generalized to characteristics of individual animals such as age, reproductive condition, body mass, etc. The first multistate models to be widely used allowed capture and survival parameters to depend on age, for studies in which sampling occasions were separated by time intervals that corresponded to the exact interval required for an animal to make the transition from 1 age class to the next (Manly & Parr, 1968; Pollock, 1981; Stokes, 1984). These age-specific models are much simpler than the general models of Arnason (1972, 1973) because of the deterministic, unidirectional nature of age transitions.

In the general multistate models of Arnason (1972, 1973), state transitions are stochastic, necessitating additional new parameters for transitions between states (also see Hestbeck et al., 1991; Brownie et al., 1993; Schwarz et al., 1993). Define \(\theta _{t}^{rs}\) as the probability that an animal in state r at occasion t that survives until occasion t + 1 is in state s at t + 1. Define \(S_{t}^{r}\) as the probability that an animal in state r at sampling occasion t is still alive and in the sampled population at occasion t + 1, and \(p_{t}^{r}\) as the probability that an animal in state r at occasion t is captured at t. Capture histories must now indicate the state of the animal at each capture. In a study area with two locations, 1 and 2, a capture history of 0102 would indicate an animal first captured in state/location 1 at sampling occasion 2, not captured at occasion 3, and captured in state 2 at occasion 4. The number of animals showing this history is denoted as x 0102, and the probability that an animal released in occasion 2, state 1, will exhibit this history and thus appear in this statistic is:

$$\displaystyle \begin{aligned} \begin{array}{rcl} & &\displaystyle \Pr(0102| \mbox{release in state 1 at occasion 2}) \\ & = &\displaystyle S_{2}^{1} [ (1 - \theta_{2}^{12}) (1 - p_{3}^{1}) S_{3}^{1} \theta_{3}^{12} + \theta_{2}^{12} (1 - p_{3}^{2}) S_{3}^{2} (1 - \theta_{3}^{21}) ] p_{4}^{2}. {} \end{array} \end{aligned} $$
(5)

The portion of expression (5) in brackets reflects the state uncertainty of the animal at occasion 3 and can be viewed as a mixture model incorporating the possibilities that the animal was in state 1 or state 2. Likelihoods are conditional on new releases in each state in each sampling occasion.

These multistate models assume the ability to classify an animal to its appropriate state without error at each capture, and they have been generalized to deal with state uncertainty and misclassification (Kendall et al., 2003, 2004; Nichols et al., 2004; Pradel, 2005). These generalizations include additional classification parameters and sometimes use ancillary data to reduce uncertainty in modeling the capture and classification processes.

2.5 Occupancy Models, Closed System, Single State

Occupancy models extend the thinking underlying capture-recapture from individual animals to a set of locations or sites. The question for a single site is whether a focal species is present or not, and the objective of the modeling is to estimate occupancy, the probability that a site is occupied by the focal species. The motivation for these models is possible nondetection; surveys of sites sometimes “miss” detecting a species, despite presence of the species at the site. A key distinction between the occupancy problem for sites and the capture-recapture problem for individuals within a population is that the number of sites is known and sites can be sampled at every occasion, although the result of the sampling is still characterized by the uncertainty of possible nondetection. Early versions of occupancy models were developed by Geissler and Fuller (1987), Azuma et al. (1990), Nichols and Karanth (2002), and most current modeling is based on MacKenzie et al. (2002).

Sample units for occupancy studies may be naturally occurring units such as ponds or woodlots, or they may be cells in a grid superimposed on a continuous area. Each unit is surveyed on multiple occasions within a relatively short time period (e.g., 2 weeks) over which there are no changes in occupancy. Detection histories are analogous to capture histories and denote the sequence of detections and nondetections at each site. The statistics resulting from such a study are the numbers of sites exhibiting each possible detection history, e.g., x 101 is the number of sites at which the species was detected on sampling occasions 1 and 3 of a 3-occasion study, but not occasion 2.

The modeling of the detection history data is similar to that for individual animal capture-recapture as well. Define p t as the probability of detecting the focal species at a sample unit on sample occasion t, and ψ as the probability that a sample unit is occupied by the species. The probability that a surveyed sample unit shows detection history 101 is (MacKenzie et al., 2002):

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \Pr(101) = \psi p_{1} (1 - p_{2}) p_{3}. \end{array} \end{aligned} $$
(6)

The probability for a site at which the species was not detected in any of the three surveys is:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \Pr(000) = \psi (1 - p_{1}) (1 - p_{2}) (1 - p_{3}) + (1 - \psi). \end{array} \end{aligned} $$
(7)

We know the species was present for detection history 101, as it was detected, and we assume no false positives (Eq. 6). However, history 000 admits more uncertainty, as there are two possibilities: the species was present and not detected or the species was absent (Eq. 7).

2.6 Occupancy Models, Open System, Single State

Open systems are those for which changes in occupancy status of sites may occur between some sample occasions. Define a primary sample occasion as a relatively short period (e.g., a specific month each year) during which occupancy status of a site is not likely to change. Multiple secondary samples (e.g., four survey days) occur within each primary period. However, the sites are permitted to be open to changes in occupancy between primary periods. For a study with three secondary occasions within each of two primary occasions, a detection history of 101,000 denotes a site with detections at secondary occasions 1 and 3 of primary occasion 1 and no detections in any of the three secondary occasions of primary occasion 2. Barbraud et al. (2003) developed an early model for such data, and the approach of MacKenzie et al. (2003) is the basis for most current modeling.

The modeling of detection probability requires an extra subscript for the two kinds of sampling occasions. Let p tk denote the detection probability associated with secondary period k of primary period t. The possibility of changes in site occupancy requires two new parameters: ε t is the probability that a site is unoccupied by the species at primary occasion t + 1, given that it was occupied at occasion t (local extinction); γ t is the probability that a site is occupied by the species at occasion t + 1, given that it was not occupied in period t (local colonization). The probability associated with the above detection history is thus:

$$\displaystyle \begin{aligned} \Pr(101~000) = \psi_{1}(p_{11} (1 - p_{12}) p_{13}) [\varepsilon_{1} + (1 - \varepsilon_{1})(1 - p_{12})(1 - p_{22})(1 - p_{23})]. \end{aligned} $$
(8)

The portion of (8) in brackets reflects the uncertainty about whether the species went locally extinct at the site or instead persisted but went undetected. The likelihood is then the product of these probabilities for the detection histories of all sites.

2.7 Occupancy Models, Multiple States, False Positives

Sometimes we may want to characterize occupied sites by “state,” where state carries additional information about an occupied site. A common situation is where a site occupied by a species can be classified into multiple states that can be ordered by the degree of uncertainty characterizing the state classification (Royle, 2004; Royle & Link, 2005, Nichols et al., 2007, MacKenzie et al., 2009). For example, assume interest in a species and an associated pathogen, such that we designate state 0 as a site not occupied by the focal species, and state 1 as a site occupied by the species but where no individuals of the species have been infected by the pathogen. State 2 denotes occupancy, with pathogen infection of at least one member of the species. In addition to these three true states, we define three observation states that can apply to a site at any secondary occasion survey: 0 =  no detection of the species; 1 =  detection of the species, but no detection of the pathogen; and 2 =  detection of both the species and pathogen. Observation state 0 admits the most uncertainty, as true state may be 0, 1, or 2. For observation state 1, true state may be 1 or 2. Under the assumption of no false-positive errors, observation state 2 is unambiguous, only occurring when true state =  2. The notation and modeling of multistate occupancy become increasingly complex (see MacKenzie et al., 2009, 2018).

The initial development of occupancy modeling assumed no false positives, where these refer to the investigator claiming to detect a species, when the species is actually absent from the sample unit. False positives typically occur when the investigator mistakes an individual or sign of one species for that of another. For example, the pugmark (track) or scat of a large leopard may be mistakenly recorded as that of a tiger. Royle and Link (2006) developed a general, single-season occupancy model that incorporates both nondetection and false positives. Miller et al. (2011) developed models that use two (or more) different detection methods to deal with false positives, and these have been extended to multiple designs (Chambert et al., 2015) and multiple seasons (Miller et al., 2013; MacKenzie et al., 2018).

2.8 Software

Computations for capture-recapture estimation of focal parameters and their variance-covariance structures are relatively complex, such that development of software has been critical to the use of these methods. Early software focused on specific parameterizations of capture-recapture models, whereas development of numerical differentiation algorithms has led to more flexible software, permitting inference for user-specified models. A variety of software packages now exists for implementing capture-recapture analyses. For example, one website (https://www.capturerecapture.co.uk/software.html) managed by R. McCrea provides links to a number of available capture-recapture packages.

Program MARK (White & Burnham, 1999; Cooch & White, 2022) implements closed and open capture-recapture models, occupancy models, and a variety of other models useful for inferences about demographic parameters. Program PRESENCE (Hines, 2006) was developed specifically for occupancy models. Historically, PRESENCE has incorporated new classes of occupancy models before other occupancy software. Program M-SURGE (Choquet et al., 2004) was developed to implement multistate capture-recapture models and is based on sufficient statistics, resulting in typically faster computation times than software such as MARK, which is based on individual capture history data. Program E-SURGE (Choquet et al., 2012) provides a general analytic framework for implementing multistate models in the presence of state uncertainty.

2.9 Summary

Both capture-recapture modeling and closely related occupancy modeling have undergone substantial evolution since their initial development for relatively simple inference problems. Most of this development has been motivated by scientists investigating animal populations and has focused on extensions and generalizations to either estimate additional parameters (i.e., beyond abundance and occupancy) or relax restrictive assumptions.

3 Beyond Traditional Applications

There have been many nontraditional uses of capture-recapture thinking and methodology. Reviews of social science and medical applications include Bohning (2008), Chao (2014), Bird and King (2018), and Bohning et al. (2018). The applications discussed in this chapter are not exhaustive but are illustrative of the diverse estimation problems to which these methods have been applied. Most of these nontraditional uses begin with a focus on abundance of some focal entity, combined with a recognition that the entity is frequently undercounted using the standard survey methods of the discipline.

3.1 Human Health and Epidemiology

3.1.1 Population-Level Inferences

Uses of capture-recapture models for human health applications have a fairly long history, with key early contributions by Wittes and Sidel (1968), Fienberg (1972), Wittes (1974), Wittes et al. (1974), Hook et al. (1980), Hook and Regal (1982, 1992), LaPorte et al. (1992), McCarty et al. (1993), and LaPorte (1994) and useful reviews by IWGDMF (1995a, 1995b), Hook and Regal (1995, 1999), and Chao et al. (2001). Virtually, all of these epidemiological uses are based on data from incomplete lists.

Some capture-recapture applications focus on single lists consisting of frequency distributions of encounters. A single list might include the number of infected individuals for which there was a single recorded encounter (e.g., blood test result, hospital visit), the number with exactly two encounters, three encounters, etc., with the objective to estimate the number of infected individuals that were never encountered. For example, Polonsky et al. (2018) used single list data to estimate the completeness, and thus effectiveness, of contact tracing.

Multiple list data are typically records of individuals infected with a particular disease or mortalities associated with a specific disease or other cause. In closed populations, the appearance of some individuals on one list and not another is clear evidence of nondetection, and early uses of multiple lists entailed first matching names that appear on multiple lists and then counting the total number of unique individuals. This approach does not include in the total count the number of individuals appearing on none of the lists, and inference about this number motivates the use of capture-recapture.

Multiple list data are encoded as individual capture or detection histories (Sects. 2.1 and 2.2), and the entire data set includes a detection history for every individual appearing on at least one list. For example, Hook et al. (1980) analyzed a data set consisting of three lists of individuals with spina bifida in New York state, 1969–1974. The lists were based on (1) birth certificates, (2) death certificates, and (3) medical rehabilitation records. Closed capture-recapture models were then used to estimate the total number of cases and disease “prevalence,” defined as the proportion of individuals in a population that is infected, or as the probability that a randomly selected individual in a population is infected. Numerous applications of capture-recapture to inferences about numbers of cases and prevalence now exist in the scientific literature.

Multiple list data differ from animal capture data in several ways that must be considered when selecting or developing capture-recapture models for epidemiological uses. The multiple lists are analogous to the multiple sampling periods of the animal ecologist, but unlike these animal sampling periods, there is frequently no natural temporal ordering of list data. Time-specificity of capture probabilities corresponds to list-specificity of detection probabilities. Certain kinds of behavioral response models in capture-recapture are based on temporal order of sampling occasions, and models (e.g., log-linear) for list data have been developed for more general kinds of dependence of detection probabilities for individuals among the different lists (see IWGDMF, 1995a, 1995b; Hook and Regal, 1995; Chao et al., 2001; Rivest and Lvesque, 2001).

Heterogeneous capture probabilities are sometimes associated with identifiable covariates, permitting inference based on a general Horvitz-Thompson approach (e.g., Huggins, 1989, 1991; also see Wang et al., 2006). Several approaches have been developed for the more difficult problem of heterogeneous capture probabilities that cannot be readily associated with covariates (e.g., Burnham & Overton, 1978; Chao, 1987; Norris & Pollock, 1995; Haas & Stokes, 1998; Dorazio & Royle, 2003; Haas et al., 2006). Problems deciding whether two similar records really match (represent the same individual) can occur when constructing detection histories from lists, and approaches for dealing with this problem (e.g., Seber et al., 2000; Lee et al., 2001) are similar, in some ways, to approaches for dealing with tag loss (e.g., Arnason & Mills, 1981; Kremers, 1988; Nichols & Hines, 1993) and misreading (e.g., McClintock et al., 2014).

Open capture-recapture models are used to estimate survival rates, numbers of new recruits, and abundance for populations open to gains and losses across multiple sampling occasions. In the context of disease dynamics, multistate models for open populations can be especially useful, where states are defined, for example, as susceptible, infected, and recovered (SIR), a classification system used in classical compartmental disease models (Kermack & McKendrick, 1927; Bailey, 1975; Cooch et al., 2012). List data based on hospital visits or longitudinal data from studies with imperfect follow-up can be used to develop detection histories for such analyses, and estimated parameters include probabilities of state transition (e.g., the transition from susceptible to infected) and state-specific mortality rates. A feature of multistate capture-recapture models that is especially important for epidemiological uses is state-specific detection probabilities (e.g., infected individuals will typically have higher probabilities of detection for hospital lists than susceptible individuals). Uncertainty in state assignment (e.g., a false-negative or false-positive pathogen test result) led to the development of models to deal with this issue (reviewed by Lebreton et al., 2009), and the multi-event approach of Pradel (2005) provides a general approach to this problem (Conn & Cooch, 2009; Choquet et al., 2013; Benhaiem et al., 2018). Multistate capture-recapture models have been recommended for use in estimating epidemiological state transition probabilities and mortality rates (Jennelle et al., 2007; Cooch et al., 2012; Nichols et al., 2017), but such uses have been relatively rare (but see Viallefont & Auget, 1999) for human diseases.

Occupancy models have several potential uses for epidemiological studies. One use entails viewing individuals as the sample units and focusing on presence or absence of the disease organism (e.g., Bailey et al., 2014; MacKenzie et al., 2018). Multistate occupancy models can be used to estimate transitions (including infection rate) among SIR model states and state-specific mortality rates, as with multistate capture-recapture. Pathogen tests for a random or representative sample of individuals can be used with standard occupancy models to estimate prevalence in the case where false negatives (nondetection) are possible (e.g., Lachish et al., 2012; Nichols et al., 2021), and even infection intensity (Miller et al., 2012). Such testing programs should typically include a subset of individuals that receive multiple tests in order to deal with nondetection.

Occupancy models can also be used to model spatial dynamics of disease spread. Data are based on tests of individuals, but now the sample unit is a location (e.g., a county or city), and interest is in whether any infected individuals are present (McClintock et al., 2010; Bailey et al., 2014; MacKenzie et al., 2018). List data could come from hospital visits, and the replication required for the most general occupancy modeling could be obtained by treating each day or week as a sampling occasion. The ability to deal with imperfect detection is especially important in such studies, as detection probabilities are likely to vary among different locations (e.g., urban locations vs. rural locations far from medical centers).

3.1.2 Individual-Level Inferences

Decisions about individual treatment and quarantine depend on the same diagnostic test results that populate lists. When such tests admit false negatives and positives, it is useful to estimate the probability that a specific individual is infected, conditional on the test result(s). Define p lm as the probability that a test result (x k = l) indicates individual k to be in disease state l, given that true disease state is m. Define state z k = 1 to mean that individual k is infected and state z k = 0 to mean uninfected. Then p 11 is the probability of correctly detecting infection when present, and its complement (1 − p 11) is the probability of a false negative. Similarly, p 10 is the probability of a false positive, incorrectly declaring an individual in state z k = 0 to be infected. Because these probabilities are conditional on the unknown true state of the tested individual, statements about the probability of true infection are also conditional on the underlying pathogen prevalence, ψ. All of the above parameters (detection/classification and prevalence) can be estimated directly using single-season occupancy models (Miller et al., 2011; Chambert et al., 2015; MacKenzie et al., 2018).

The conditional probability that an individual testing positive is actually infected (“positive predictive value”) can be written as:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \Pr(z_{k} = 1 | x_{k} = 1) = \frac{\psi p_{11}}{\psi p_{11} + (1 - \psi) p_{10}}. \end{array} \end{aligned} $$
(9)

The conditional probability that an individual testing negative is truly not infected (termed “negative predictive value”) can be written similarly as:

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \Pr(z_{k} = 0 | x_{k} = 0) = \frac{(1 - \psi) (1 - p_{10})}{\psi (1 - p_{11}) + (1 - \psi) (1 - p_{10})}. \end{array} \end{aligned} $$
(10)

Note that if prevalence parameters are likely to differ for different groups of individuals (e.g., those exhibiting symptoms and those not), then group-specific prevalence parameters should be estimated and used. If the probabilities of an accurate test result [(9) and (10)] are thought to be too small for important decisions about individual treatment, then multiple tests can be used to increase them (e.g., Nichols et al., 2021).

An advantage of the occupancy approach over that frequently used by epidemiologists is that all of the relevant parameters can be estimated together in a joint likelihood. The probabilities of an individual being infected are computed directly as derived parameters, with the associated estimates of sampling variance properly accounting for the variances and covariances of the different parameter estimates.

3.2 Social Sciences

3.2.1 Census

Governments of most countries conduct periodic “censuses” of population size and distribution. However, direct counts are seldom possible, and virtually all census methods miss individuals (false negatives). Laplace (1786) was the first to derive the estimator (2) and used it to compute the human population size of France using two lists of citizens and their degree of overlap. Sekar and Deming (1949) appeared to derive the estimator (2) independently of (Laplace, 1786) and (Lincoln, 1930) and used it to draw inferences about the numbers of human births and deaths in a district near Calcutta, India. Application of capture-recapture methods (sometimes referred to as “multiple systems estimation” in the social science literature; Fienberg & Manrique-Vallier, 2009; Bird & King, 2018) to problems in the social sciences has increased in recent decades prompting methodological reviews (e.g., Bohning, 2008, Bird & King, 2018) and a book (Bohning et al., 2018).

Capture-recapture models for closed populations have been extended by scientists working with the United States Census Bureau and used with post-enumeration surveys to estimate the census undercount (Wolter, 1986, 1990; Cowan & Malec, 1986; U.S. Census, 2021). Evaluation of census coverage using post-enumeration surveys along with capture-recapture estimation has been recommended by the United Nations (Demographic and Social Statistics Branch, United Nations Statistics Division, 2009) and is being used by various countries (e.g., UK Abbott, 2009; Turkey, Ayhan & Ekni, 2003; Australia, Australian Bureau of Statistics, 2012). In addition to use with standard governmental censuses, capture-recapture approaches have been especially useful for providing inferences about “hidden” populations, groups of individuals that are especially difficult to count using conventional surveys, frequently because they do not wish to be counted (e.g., Sudman et al., 1988).

3.2.2 Homeless

Homeless persons are a problematic group for conventional governmental census methods, as they typically lack a mailing address and are not motivated to provide census information. Fisher et al. (1994) obtained list data for homeless persons from multiple sources including hospitals, local social service agencies, a healthcare center designated for homeless, and hostels and used capture-recapture to estimate the homeless population in an area of London. Their estimate was approximately three times larger than the number of list-identified individuals. Such multiple-list approaches have been used with capture-recapture modeling to estimate homeless populations elsewhere as well (e.g., Baltimore, Cowan et al., 1986; Budapest, David & Snijders, 2002). Berry (2007) used an observational approach to identify homeless individuals in Toronto on the street during multiple sampling occasions. Closed capture-recapture models were used to estimate the homeless population, and detection probabilities of about 0.2 indicated the importance of dealing with nondetection.

An alternative approach to multiple lists is to insert some number of “marked” (M) or “planted” individuals into the focal homeless population and then survey the population directly, estimating detection probabilities as the proportion of planted individuals that is detected (Eq. 3). Laska and Meisner (1993) identified 103 sites frequented by homeless persons in a region of New York City and planted persons in a random sample of 41 of these sites. Census Bureau enumerators were then sent to directly survey homeless persons at these sites. Detection probability was estimated to be 0.48 and used to estimate the total number of homeless in the surveyed areas.

3.2.3 Problem Drug Users

Capture-recapture models have been used with list data on individuals to estimate numbers of problem drug users in various locations. For example, King et al. (2014) used four list sources, probation records, drug intervention program prison assessments, drug treatment facility records, and drug intervention program community assessments, to estimate the number of injecting drug users and heroin-associated deaths in England. They used a Bayesian approach to incorporate prior information into their capture-recapture modeling, obtaining estimates for England, as well as for specific regions within the country. Both prevalence of problem drug use and detection probabilities (probability that a problem drug user appears on at least one list) showed substantial regional variation. Approaches based on similar list data were used to estimate numbers of injecting drug users in Scotland (King et al., 2013). Capture-recapture approaches to inference about problem drug use are numerous and include inferences about the number of HIV-infected injecting drug users in Bangkok (Mastro et al., 1994), prevalence of opiate use in Dublin (Comiskey and Barry, 2001), prevalence of problem drug use in London (Hickman et al., 1999) and six French cities (Vaissade & Legleye, 2008), the risk of arrest of drug dealers and users in Quebec (Bouchard & Trembley, 2005), and the number of heroin users in the Australian Capital Territory (Larson et al., 1994).

3.2.4 Criminal Activities

Greene and Stollmack (1981) applied closed population capture-recapture methods to records from approximately 6000 males arrested at least once in Washington, D.C., 1974–1975. They estimated a total criminal population of about 30,000 individual criminals. Using these same data, Greene (1984) later applied an open population model permitting inferences about growth rate of the offender population, survival probabilities, and average criminal career length. Bouchard et al. (2019) used capture-recapture with arrest and rearrest record data from Quebec to estimate the number of criminals involved with illegal amphetamine-like stimulants. They estimated that total arrests were only about 12% of those actually engaged in illegal activities and subject to arrest. Bouchard (2007) used capture-recapture methods with arrest data to estimate the number of criminal marijuana growers in Quebec, 1998–2002. Charette and van Koppen (2016) used capture-recapture methods to investigate selectivity in crime punishment, concluding that black male offenders were more likely to be arrested and punished than members of other demographic groups.

Cases of domestic violence in the Netherlands, 2006–2007, were estimated by van der Heijden (2014) using capture-recapture methods with police register records. Their estimates indicated that about 22% of offenders were actually observed and recorded by police. Silverman (2014) used capture-recapture modeling of multiple list data to estimate the number of victims of human trafficking in the UK, 2013. Data on individual victims came from six lists: local authority, police force, national government organization, nongovernment organization, National Crime Agency, and the general public. The estimated victim population was four to five times larger than the number of individuals detected.

Corlatti et al. (2019) studied illegal poaching of red deer in a park in the central Italian Alps, 2007–2017. They estimated age- and sex-specific mortality rates of deer associated with poaching and non-poaching sources using open, multi-event models with data for tagged red deer. Their modeling included parameters for tag loss and the possibility of misclassifying the cause of death (by poaching or not) and provided strong evidence of higher poaching mortality for older males than any other age-sex class.

Barber-Meyer (2010) proposed use of occupancy models with data on species (e.g., tiger parts and products) sold illegally at souvenir shops, traditional medicine stores, etc., within towns. Replication is provided by the multiple stores and shops within each town. Towns were the sample units, such that occupancy estimated the proportion of towns at which the focal species was illegally sold, and multiseason models could be used to estimate occupancy dynamics over time. Sharma et al. (2014) used reports of annual tiger poaching events reported by the Wildlife Protection Society of India, in conjunction with multiseason occupancy modeling, to estimate the prevalence of tiger poaching during periods of 3–7 years in 605 districts throughout India over a 40-year period. Results provided maps of tiger poaching crime and information about covariates associated with such crime.

Yeo et al. (2017) used eBay postings to estimate aspects of illegal elephant ivory trade dynamics in the UK. Each posting was identified by a description, item number, and seller identification, permitting identification of the item in subsequent postings. Postings were surveyed once per week for eight consecutive weeks, March–May 2014. Detection histories were developed for every item and used with open-population capture-recapture models to estimate numbers of items, as well as weekly survival (persistence in the eBay market) and entry probabilities. The authors concluded that a large fraction of illegal ivory sale items had very low probabilities of detection.

3.2.5 World Conflicts

Armed conflicts throughout the world result in numbers of persons being killed or disappearing, and “counts” of these victims are typically biased low. Capture-recapture methods have been used with casualty list data to estimate numbers of victims associated with conflicts in Peru (Ball et al., 2003; Manrique-Vallier et al., 2013) and Colombia (Lum et al., 2013); number of deaths in Kosovo, March–June 1999 (Ball & Asher, 2002); and the number of persons killed by state forces in Guatemala, 1981–1983 (Ball, 2000). For example, in the Guatemala analysis, lists of victims were provided by the following three sources, the Commission for Historical Clarification, the International Center for Human Rights Research, and the Catholic Church’s Interdiocesan Project for the Recuperation of Historical Memory. The estimated number of killings was about three times larger than the sum of victims identified via the three lists.

Social conflict events from some parts of the world are not well reported, such that counts of such events are typically biased low. Hendrix and Saleyhan (2015) used closed population capture-recapture models to estimate the number of social conflict events occurring across Africa in 2012. They obtained detection/nondetection data on 1443 events from the Social Conflict in Africa Database. They used data from two independent news agencies, Associated Press (AP) and Agence France-Presse (AFP), compiling statistics on numbers of events reported only by AP, only by AFP, and by both agencies. They concluded that these two news sources captured approximately 76% of all events in Africa and that the nondetection rate was predictably smaller for deadly events, events of a larger magnitude, and events associated with government repression.

3.3 Quality Control

Capture-recapture models have been used for several specific problems associated with quality control. Jewell (1985) noted that defects or errors can occur in production of various manufactured goods, in computer software, in manuscripts, etc. and recommended capture-recapture approaches for estimating numbers of them. Quality control efforts typically involve inspectors or proofreaders who examine products for defects or errors, but errors may go undetected. One approach to estimating number of errors/defects in the face of nondetection is to employ multiple inspectors or proofreaders. In the case of three inspectors, for example, each error detected by at least one inspector is represented by a row of three entries (one entry for each inspector), with a 1 denoting detection of the error by the particular inspector and a 0 denoting nondetection. Chao and Yang (1993) used this approach with computer code examined by multiple coders looking for errors and estimated the number of errors remaining (undetected). White et al. (1982) used this approach with multiple proofreaders of a large manuscript and estimated the number of undetected errors.

3.4 Remote Sensing

“Remote sensing” refers to use of aircraft or satellites to obtain information about the earth. Photography and video are typically used to provide images, which are then examined by individuals or computers in order to enumerate focal entities (e.g., wetlands, woodlots) or compute area measurements of specific cover types. However, such analyses of remote sensing images are usually characterized by two types of errors, nondetection and misclassification. These errors can sometimes be dealt with via replication provided by multiple observers in aircraft or multiple persons processing the same image. Capture-recapture methods are then used to estimate number of entities, for example, using the number of entities detected by just one observer, two observers, etc. (Magnusson et al., 1978; Cook and Jacobson, 1979).

More commonly, a sample of area covered by a survey is visited by ground observers providing direct counts and classifications, known as ground truthing. The number of ground truth entities that is correctly detected or classified via the remote images is then used with capture-recapture thinking to estimate detection and correct classification probabilities (see Maxim et al., 1981; Maxim & Harrington, 1982, 1983). Veran et al. (2012) focused on the question of land cover dynamics, noting that classification errors can be made at times t, t + 1, or both times, leading to large errors in estimates of land cover state transition probabilities. They proposed use of ground truth data with multistate capture-recapture models that included state misclassification as a means of directly estimating land cover transitions in the face of classification errors.

3.5 Paleobiology

Paleobiologists have long recognized that nondetection is an important issue for analyzing fossil data (e.g., Foote & Raup, 1996). Analyses that do not account for nondetection are subject to serious errors, as detection probabilities are thought not only to be substantial but also to vary across time and space (Brett, 1998). Paleobiological data consist of records of fossil taxa found via sampling at different strata (different geologic time horizons) and locations. Capture-recapture analyses typically treat each lower-level taxon (e.g., family) within a higher-level taxon (e.g., phylum) as an “individual.” Detection histories for each lower-level taxon can be developed using spatial samples (analogous to multiple lists) within some time stratum and area of interest, providing the data for estimation of total taxa using closed capture-recapture models. Detection histories for focal taxa developed from different geologic strata (time horizons) at the same sampling location, or even worldwide, can be used with open capture-recapture models to estimate number of taxa and rates of both local and global taxonomic origination and extinction for lower-level taxa (Nichols & Pollock, 1983). Capture-recapture models were introduced to paleobiology over 40 years ago (Rosenzweig & Duek, 1979; Nichols & Pollock, 1983; Conroy & Nichols, 1984; Nichols et al., 1986), but they have seen only limited use (Connolly & Miller, 2001a, 2001b, 2002).

Occupancy models have multiple uses for fossil data as well, with focus on a specific taxon, rather than a group of lower-level taxa. Detection-nondetection data from replicate local samples can be used in conjunction with occupancy models to estimate geographic distribution (Liow &Nichols, 2010; Liow, 2013). Detection histories based on different time horizons from multiple locations can be used with occupancy modeling to estimate local probabilities of colonization and extinction as well (Liow &Nichols, 2010; Liow, 2013). Occupancy models were introduced to paleobiologists much more recently than capture-recapture models (Liow &Nichols, 2010; Liow, 2013; MacKenzie et al., 2018), and paleobiological use of occupancy approaches has been limited (but see Lawing et al., 2021).

3.6 Miscellaneous Applications

National databases for traffic accidents are maintained by law enforcement agencies in many countries, but accidents are thought to be underreported, leading to many efforts to estimate their true numbers using capture-recapture. Razzak and Luby (1998) compiled lists of police accident records and emergency ambulance service records over a 10-month period during 1994 in Karachi, Pakistan. Their estimates indicated that official records accounted for 56% of traffic accident deaths and only 4% of serious injuries. Capture-recapture inferences about traffic accidents have been used in various other locations including Nicaragua (Tercero & Andersson, 2004), Ethiopia (Abegaz et al., 2014), and Mali (Sango et al., 2016).

Beirne and Lambin (2013) studied volunteer “citizen scientists” who worked on a project to remove invasive mink from a large area of Scotland. Their objective was to draw inferences about volunteer retention (tendency to remain in the program) and the factors that affected it. They described the potential utility of open capture-recapture approaches, but collected data on volunteer activity data via telephone every 6 months and were thus able to use known-fate models (Pollock et al., 1989). They identified volunteer vocation and recent trapping and removal success as key determinants of retention in the program.

Interest in vocabulary size has prompted literary scholars to count the number of individual words that an author uses in her/his writing, but this number is likely smaller than the number actually known to the author. Efron and Thisted (1976) counted the number of words used once, twice, three times, etc., in samples of Shakespeare’s writing in order to estimate the total number of words that he knew. Words counted in the samples totaled 31,534, and capture-recapture estimators indicated that he knew about 35,000. Capture-recapture methods have also been used to estimate the song repertoire size of birds (Garamszegi et al., 2002).

An archaeological use of capture-recapture modeling was provided by Holst (1981), with subsequent reanalyses using different capture-recapture estimators by Esty (1982, 1983) and Chao (1984). The problem was to estimate the number of different “dies” that produced a set of 204 coins in ancient India. The data were the number of dies that produced only a single coin in the sample, two coins in the sample, three, etc.

Herendeen and White (2013) collected data on appearances of specific rare stamps over the years from sources such as auction catalog, retail price lists, copies of expert certificates, and similar records. Each stamp has an individual identifying number. Herendeen and White (2013) viewed each year as a sampling occasion and used closed population models to estimate the total number of stamps still in existence.

Nichols et al. (2013a, 2013b) used capture-recapture thinking to estimate detection and classification probabilities for military imaging systems. Vessels of different classes (defined by size, and military vs. civilian status) were experimentally positioned at different distances from two new cameras. “No vessel” was one of the experimental possibilities as well. Resulting data were used to develop a model for detection and classification probabilities as a function of distance and vessel type. Capture-recapture model selection was used to infer that distance relationships were dependent on camera type but characterized by a common slope across vessel types (Nichols et al., 2013b).

K.H. Pollock (pers. comm.) used capture-recapture models to estimate the number of man-made objects orbiting earth. The field of astronomy is characterized by substantial nondetection, with detection probabilities a function of telescope type as well as distance, brightness, and size of focal object, and a number of potential uses of capture-recapture thinking can be envisaged.

4 Discussion

The problems of nondetection and misclassification characterize numerous types of count data. The various applications described in Sect. 3 have hopefully supported this assertion, and there are certainly many more applications that can be imagined. The adoption of capture-recapture thinking has not been as rapid as might be hoped for any of the disciplines of Sect. 3, and rate of adoption has varied among these disciplines. For example, my impression based on literature review is that epidemiological and human health applications are somewhat more common than those dealing with social sciences, whereas adoption within paleobiology has been very slow.

One possible explanation underlying this variation in rate of adoption involves the perceived severity of nondetection and misclassification (Nichols, 2019). For example, the development of capture-recapture thinking in the fields of wildlife and animal ecology is likely a natural response to the well-known nondetection problems associated with virtually all animal survey methods. Not only do surveys miss animals, but the fraction missed can be very large. In contrast, epidemiological and social science data based on counts of humans have historically been thought to be closer to truth, although this perception is changing. Indeed, some of the detection probability estimates of Sect. 3 are quite small.

A second factor that may affect rate of adoption of capture-recapture is the cost of incorrect counts and estimates, and thus the scrutiny that analytic results receive. Results of epidemiological surveys and medical diagnoses are viewed as extremely important and are often carefully reviewed, as misleading inferences can have detrimental consequences that may be readily apparent. In contrast, inferences in the social sciences are certainly important, but not so highly scrutinized, and misleading inferences are less likely to be recognized. The greater the degree of scrutiny, the greater the expected attention to analytic details and inferential errors.

Another factor that may affect rate of adoption is the funding available to a discipline. Good funding helps ensure collaboration of statisticians, who are able to deal with the added complexity of modeling sampling processes. Epidemiology and human health are among the better funded scientific disciplines in most countries.

One more factor influencing methodological adoption is likely the familiarity of scientists with capture-recapture approaches. My search for uses of capture-recapture models for inferences about criminal activity produced a number of papers that used capture-recapture to study the criminal activity of animal poaching. I am guessing that the appearance of disproportionate numbers of applications for this particular type of crime resulted from prior familiarity of scientists investigating such crimes with capture-recapture approaches.

This relatively slow adoption of robust methods for dealing with nondetection and misclassification begs the question: what are the alternatives to modeling these components of the sampling process? The most common alternative appears to be to view the problems as so small and insignificant that they can be safely ignored. For example, this has been the case with remote sensing uses, as ground truthing data have provided clear evidence of nondetection and misclassification. However, estimates of these errors are frequently presented, claimed to be small, and then ignored in analysis (see discussion in Veran et al., 2012). I suspect that this alternative is also prevalent in disciplines where errors are not so readily estimated, but rather assumed or claimed to be small and thus not worthy of the effort to deal with nondetection.

A second alternative to the use of capture-recapture is to try to identify the key sources of variation in detection probability or misclassification, to develop models for each of these component processes separately, and then to combine these models to provide overall inferences about detection probability and the focal parameters that they influence. I encountered this approach at a 2011 workshop dealing with nondetection. A biostatistician working for the Centers for Disease Control and Prevention outlined this approach to inference about detection probability for a focal disease. Her strategy was to develop a model for each of 10–12 sources of variation in detection and to then combine these models into an overall model for the sampling process. At the time of the workshop, two of these models had been developed. In contrast to this incremental approach, capture-recapture requires some form of replicate sampling (e.g., multiple lists) and then directly uses the information about nondetection available in detection history data. Detection probability, as well as focal parameters such as numbers of cases or disease prevalence, is estimated without the need for identification and modeling of all factors affecting nondetection. If potential influencing factors can be identified, then they can be treated as covariates in capture-recapture modeling, with the result that their influence can be formally tested and, if found to be important, included in the modeling of detection probability.

A problem related to slow adoption of capture-recapture approaches is the limited inferences to which they are applied. The early history of capture-recapture in wildlife and animal ecology was dominated by a focus on numbers. Capture-recapture estimators for closed populations were used to provide estimates of population size for specific locations and times. However, abundance is not necessarily interesting by itself, but rather is more usefully viewed as a state variable in studies of dynamical processes. The primary interests are sources of spatial and temporal variation in abundance, and the ability of human actions to influence population size. This recognition eventually led to increased interest in capture-recapture models for open populations that experience dynamical changes between sampling occasions. Today’s capture-recapture studies of wildlife populations tend to focus on the processes of birth, death, and movement, and on transition probabilities governing changes of state within individuals. Similarly, occupancy estimates themselves are not viewed as especially interesting, and focus has shifted to the probabilities of local extinction and colonization that govern occupancy dynamics.

This review of nontraditional applications of capture-recapture models suggests to me that most of these studies are focusing on numbers of focal entities. As noted in Sect. 3.1.1, epidemiological list data from hospital visits and data from longitudinal studies with incomplete follow-up can be used with multistate capture-recapture models for open populations to draw inferences about state transition probabilities and state-specific mortality rates required by SIR (susceptible, infected, recovered) models. These models provide a way to deal with the state-specific detection probabilities likely to exist in longitudinal data. For example, an individual in the infected state at sampling occasion t is more likely to be found on a hospital list or re-encountered in a longitudinal study at that occasion than an individual in the susceptible or recovered state. Accompanying information on public health interventions or even individual treatments can be used with these models to directly test the efficacy of interventions and treatments. Despite the potential utility of multistate (Lebreton et al., 2009) and multi-event (Pradel, 2005) capture-recapture models, I saw little evidence that these approaches are being used in disciplines other than animal and wildlife ecology.

The primary interest of most capture-recapture applications in criminology (Sect. 3.2.4) was in numbers of criminals or victims, or in detection probability when this equated with probability of arrest. These studies were not focused on the influences of laws or enforcement interventions on criminal activity, or on the effects of crimes on victims. Contrast this with a number of the studies of wildlife poaching crimes which included investigations of poaching-related mortality rates (Corlatti et al., 2019), and of relationships between dynamics of poaching activity and species distributions (Marescot et al., 2019; Moore et al., 2021), and between ranger (enforcement) activity and poaching activity (Moore et al., 2017). I suspect that a main reason for this difference in uses of capture-recapture for wildlife vs. other crimes stems from the familiarity of persons focused on wildlife crimes with these more complicated models and their utility.

With the exception of these investigations of wildlife crimes, the majority of the nontraditional uses of capture-recapture have focused on numbers. Such studies can be very useful when these numbers are incorporated into a larger sampling scheme designed to test hypotheses about system dynamics or effects of potential interventions. However, for most of the reviewed papers describing nontraditional uses of capture-recapture, this was not the case. I believe that studies that go beyond estimates of numbers to focus on system dynamics and key relationships (e.g., effects of interventions or treatments) are much more likely to be useful to both science and decision-making. Capture-recapture and occupancy models developed for open populations are especially useful for investigating underlying processes, and I would hope that we see the same increases in use of these models in nontraditional applications as we did in the fields of wildlife and animal population ecology.

In summary, studies in many disciplines are based on count data, yet counts are frequently inaccurate because of nondetection and misclassification. The fields of wildlife and animal ecology recognized these problems nearly a century ago and began to develop capture-recapture approaches to deal with them. Other disciplines began to adopt these methods and, in some cases, to modify them for their specific applications (e.g., log-linear models for closed populations to deal with list dependence). However, the integration of capture-recapture methods into the toolboxes of scientists of non-ecological disciplines has been incomplete and slower than might be hoped. In addition, the bulk of non-traditional uses of capture-recapture models has focused on estimation of totals for counted entities. I hope that use of capture-recapture models for nontraditional applications increases and that such uses better exploit the open-population models that permit inferences about dynamical processes.