Harvesting of interacting stochastic populations

Abstract

We analyze the optimal harvesting problem for an ecosystem of species that experience environmental stochasticity. Our work generalizes the current literature significantly by taking into account non-linear interactions between species, state-dependent prices, and species seeding. The key generalization is making it possible to not only harvest, but also ‘seed’ individuals into the ecosystem. This is motivated by how fisheries and certain endangered species are controlled. The harvesting problem becomes finding the optimal harvesting-seeding strategy that maximizes the expected total income from the harvest minus the lost income from the species seeding. Our analysis shows that new phenomena emerge due to the possibility of species seeding. It is well-known that multidimensional harvesting problems are very hard to tackle. We are able to make progress, by characterizing the value function as a viscosity solution of the associated Hamilton–Jacobi–Bellman equations. Moreover, we provide a verification theorem, which tells us that if a function has certain properties, then it will be the value function. This allows us to show heuristically, as was shown by Lungu and Øksendal (Bernoulli 7(3):527–539, 2001), that it is almost surely never optimal to harvest or seed from more than one population at a time. It is usually impossible to find closed-form solutions for the optimal harvesting-seeding strategy. In order to by-pass this obstacle we approximate the continuous-time systems by Markov chains. We show that the optimal harvesting-seeding strategies of the Markov chain approximations converge to the correct optimal harvesting strategy. This is used to provide numerical approximations to the optimal harvesting-seeding strategies and is a first step towards a full understanding of the intricacies of how one should harvest and seed interacting species. In particular, we look at three examples: one species modeled by a Verhulst–Pearl diffusion, two competing species and a two-species predator–prey system.

1 Introduction

Real populations never evolve in isolation. As a result, a key question in ecology is finding conditions that allow multiple species to coexist. There is a general theory for deterministic coexistence (Hofbauer 1981; Hutson 1984; Hofbauer and So 1989; Hofbauer and Sigmund 1998; Smith and Thieme 2011). However, due to the intrinsic randomness of environmental fluctuations, deterministic models should be seen only as first order approximations of the real world. In order to get a better understanding of population dynamics we have to take into account environmental stochasticity. Recently there has been significant progress towards a general theory of stochastic coexistence (Schreiber et al. 2011; Benaim 2018; Benaïm and Schreiber 2018; Hening and Nguyen 2018).

Many species of animals live in restricted habitats and are at risk of being overharvested. Harvesting, hunting and other forms of overexploitation have already driven species to extinction. On the other hand, underharvesting can lead to the loss of valuable resources. One has to carefully balance both conservation and economic considerations in order to find the optimal harvesting strategies. It can take a population a significant amount of time to recover from large harvests. This, in combination with the random environmental fluctuations, can make it impossible for the population to survive and can lead to extinctions (Lande et al. 1995, 2003).

In certain cases, added conservation efforts have to be made in order to save a species from extinction. Therefore, it makes sense to be able to repopulate a species by seeding animals into the habitat. There is no reason to assume that the price of the harvesting or seeding is constant. If the harvested population is smaller the cost of harvesting is usually higher due to the fact that it is harder to find the individuals one wants to harvest. Similarly, the marginal cost of seeding will be lower, if one has a large population. We present a model that incorporates all these factors and effects. We consider $d\ge 0$ species interacting nonlinearly in a stochastic environment where the species can be harvested as well as seeded into the system and the price of harvesting and seeding is density-dependent. The problem becomes finding the optimal harvesting-seeding strategy that maximizes the expected total income from the harvest minus the lost income from the species seedings. Mathematically, the problem we consider belongs to a class of singular stochastic control problems. Singular stochastic control problems have been studied extensively in various settings. To mention just a few, we refer to works of Alvarez and Shepp (1998), Alvarez (2000), Lungu and Øksendal (1997), Song et al. (2011), Hening et al. (2019), Alvarez and Hening (2018) for single species ecosystems in random environments and of Lungu and Øksendal (2001), Tran and Yin (2015), Tran and Yin (2017) for interacting populations. The reader can also find analogous results in the setting of corporate strategy (Radner and Shepp 1996), and optimal dividend strategies (Asmussen and Taksar 1997; Jin et al. 2013; Scheer and Schmidli 2011). Numerical methods for optimal harvesting have been developed by Jin et al. (2013), Tran and Yin (2016) and capital injections have been introduced by Dickson and Waters (2004), Kulenko and Schmidli (2008), Scheer and Schmidli (2011).

Considering optimal dividend problems in insurance and risk management (Dickson and Waters 2004; Jin et al. 2013; Kulenko and Schmidli 2008; Scheer and Schmidli 2011), it was observed that higher profit can be obtained if investors are allowed not only to remove but also to inject capital. In the harvesting setting, the idea of repopulating species (which we will call seeding) is natural and has been done for conservation efforts as well as for fisheries and agriculture. We propose a general model in which the control consists of two components: harvesting and seeding. In contrast to the existing literature, in our framework, to maximize the expected total discounted reward, the controller can add individuals of various species to maintain the system at a certain level and to avoid extinction. Moreover, we work with a system of interacting species. There are few theoretical results regarding the multi-species harvesting problem (Lungu and Øksendal 2001; Tran and Yin 2017). In a model with several species, one needs to decide which species to harvest at a given time. In addition, our model is complicated because we also allow seeding. At a given time, there are several possibilities. One can do nothing and let the population dynamics run on its own, or one can have any possible combination of seeding and harvesting of the d species.

To find the optimal harvesting-seeding strategy (also called the optimal control) and its associated total discounted reward (also called the value function), the usual approach is to solve the associated the Hamilton–Jacobi–Bellman (HJB) partial differential equations. However, for the singular control problems we consider, the HJB equations become a system of nonlinear quasi-variational inequalities. We use the viscosity solution approach for partial differential equations to study the value functions and associated control problems. It is usually impossible to find closed-form solutions to the HJB system. In order to side-step this difficulty and still gain valuable information, we develop numerical algorithms to approximate the value function and the optimal harvesting-seeding strategy. We do this by using the Markov chain approximation methodology developed by Kushner and Martins (1991).

The main contributions of our work are the following:

• (1) We formulate the harvesting-seeding problem for a system of interacting species living in a stochastic environment.

• (2) We establish the finiteness and the continuity of the value function and characterize the value function as a viscosity solution of an associated HJB system of quasi-variational inequalities.

• (3) We develop numerical approximation schemes based on the Markov chain approximation method.

• (4) We discover new phenomena by analyzing natural examples for one and two-species systems.

The rest of our work is organized as follows. In Sect. 2 we describe our model and the main results. Particular examples are explored using the newly developed numerical schemes in Sect. 3. Finally, all the technical proofs appear in the appendices.

2 Model and results

Assume we have a probability space $(\Omega , {\mathcal {F}}, {{\mathbb {P}}})$ satisfying the usual conditions. We consider d species interacting nonlinearly in a stochastic environment. We model the dynamics as follows. Let $\xi _i(t)$ be the abundance of the ith species at time $t\ge 0$, and denote by $\xi (t)=(\xi _1(t), \dots , \xi _d(t))'\in \mathbb {R}^d$ (where $z'$ denotes the transpose of z) the column vector recording all the species abundances.

One way of adding environmental stochasticity to a deterministic system is based on the assumption that the environment mainly affects the growth/death rates of the populations. This way, the growth/death rates in an ODE (ordinary differential equation) model are replaced by their average values to which one adds a white noise fluctuation term (Turelli 1977; Braumann 2002; Gard 1988; Evans et al. 2013; Schreiber et al. 2011; Gard 1984).

Under this assumption the dynamics becomes

$$\begin{aligned} d \xi (t)=b(\xi (t)) dt+\sigma (\xi (t)) d w(t). \end{aligned}$$

(2.1)

where $w(\cdot )=(w_1(\cdot ), ..., w_d (\cdot ))'$ is a d-dimensional standard Brownian motion and $b,\sigma :[0,\infty )^d\rightarrow [0,\infty )^d$ are smooth enough functions. Let $S=(0, \infty )^d$ and ${{\bar{S}}}=[0, \infty )^d$. We assume that $b(0)=\sigma (0)=0$ so that 0 is an equilibrium point of (2.1). This makes sense because if our populations go extinct, they should not be able to get resurrected without external intervention (like a repopulation/seeding event). If $\xi _i(t_0)=0$ for some $t_0\ge 0$, then $\xi _i(t)=0$ for any $t\ge t_0$. Thus, $\xi (t)\in {{\bar{S}}}$ for any $t\ge 0$.

For $x, y\in \mathbb {R}^d$, with $x=(x_1, \dots , x_d)'$ and $y=(y_1, \dots , y_d)'$, we write $x\le y$ or $y\ge x$ if $x_j\le y_j$ for each $j=1, \dots , d$, while $x < y$ if $x_j < y_j$ for each $j=1, \dots , d$.. We also define the scalar product $x\cdot y=\sum _{j=1}^d x_j y_j$. For a real number a, we denote $a^+=\max \{a, 0\}$ and $a^-=\max \{-a, 0\}$. Thus, $a=a^+-a^-$ and $|a|=a^++a^-$. For $x=(x_1, \dots , x_d)'\in \mathbb {R}^d$, $x^+=\big ( x_1^+, \dots , x_d^+\big )'$ and $x^-=\big ( x_1^-, \dots , x_d^-\big )'$. Let $\mathbf{e_i}\in \mathbb {R}^d$ denote the unit vector in the ith direction for $i=1, \dots , d$.

To proceed, we introduce the generator of the process $\xi (t)$. For a twice continuously differentiable function $\Phi (\cdot ): \mathbb {R}^d\mapsto \mathbb {R}$, we define

$$\begin{aligned} \begin{array}{ll} {\mathcal {L}}\Phi (x)&\!\displaystyle \,=b(x)\nabla \Phi (x)+\dfrac{1}{2}\hbox {tr}\big (\sigma (x)\sigma '(x)\nabla ^2 \Phi (x)\big ), \end{array} \end{aligned}$$

where $\nabla \Phi (\cdot )$ and $\nabla ^2 \Phi (\cdot )$ denote the gradient and Hessian matrix of $\Phi (\cdot )$, respectively.

Next, we have to add harvesting and seeding to (2.1). Let $Y_i(t)$ denote the amount of species i that has been harvested up to time t and set $Y(t)=(Y_1(t), \dots , Y_d(t))'\in \mathbb {R}^d$. Let $Z_i(t)$ denote the amount of species i seeded into the system up to time t and set $Z(t)=(Z_1(t), \dots , Z_d(t))'\in \mathbb {R}^d$. The dynamics of the d species that takes into account harvesting and seeding is given by

$$\begin{aligned} X(t)=x+\int \limits _0^t b(X(s)) ds + \int \limits _0^t \sigma (X(s)) dw(s) - Y(t) +Z(t), \end{aligned}$$

(2.2)

where $X(t)=(X_1(t), \dots , X_d(t))'\in \mathbb {R}^d$ are the species abundances at time $t\ge 0$. We also assume the initial species abundances are

$$\begin{aligned} X(0-)=x\in {{\bar{S}}}. \end{aligned}$$

(2.3)

Notation For each time t, $X(t-)$ represents the state before harvesting starts at time t, while X(t) is the state immediately after. Hence X(0) may not be equal to $X(0-)$ due to an instantaneous harvest Y(0) or an instantaneous seeding Z(0) at time 0. Throughout the paper we use the convention that $Y(0-)=Z(0-)=0$. The jump sizes of Y(t) and Z(t) are denoted by $\Delta Y(t):=Y(t) - Y(t-)$ and $\Delta Z(t):=Z(t) - Z(t-)$, respectively. We use $Y^c(t):= Y(t) - \sum \nolimits _{0\le s\le t} \Delta Y(s)$ and $Z^c(t):= Z(t) - \sum \nolimits _{0\le s\le t} \Delta Z(s)$ to denote the continuous part of Y and Z. Also note that $\Delta X(t):= X(t) - X(t-)= \Delta Z(t)-\Delta Y(t)$ for any $t \ge 0$.

Let $f_i: {{\bar{S}}}\mapsto (0, \infty )$ represent the instantaneous marginal yields accrued from exerting the harvesting strategy $Y_i$ for the species i, also known as the price of species i. Let $g_i: {{\bar{S}}}\mapsto (0, \infty )$ represent the total cost we need to pay for the seeding strategy $Z_i$ on species i. We will set $f=(f_1, \dots , f_d)'$ and $g=(g_1, \dots , g_d)'$. For a harvesting-seeding strategy (Y, Z) we define the performance function as

$$\begin{aligned} J(x, Y, Z):= {{\mathbb {E}}}_{x}\bigg [\int \limits _0^{\infty } e^{-\delta s} f( X(s-) )\cdot dY(s)-\int \limits _0^{\infty } e^{-\delta s} g( X(s-)) \cdot dZ(s)\bigg ],\nonumber \\ \end{aligned}$$

(2.4)

where $\delta > 0$ is the discounting factor, ${{\mathbb {E}}}_{x}$ denotes the expectation with respect to the probability law when the initial abundances are $X(0-)=x$, and $f(X(s-))\cdot dY(s):=\sum _{i=1}^d f_i(X(s-)) dY_i(s)$.

Control strategy Let ${\mathcal {A}}_{x}$ denote the collection of all admissible controls with initial condition x. A harvesting-seeding strategy (Y, Z) will be in ${\mathcal {A}}_x$ if it satisfies the following conditions:

(a):

the processes Y(t) and Z(t) are right continuous, nonnegative, and nondecreasing with respect to t; moreover, $\Delta Y_i(t) \Delta Z_i(t)=0$ for $i=1, \dots , d$ and $t\ge 0$,

(b):

the processes Y(t) and Z(t) are adapted to $\sigma \{w(s): 0\le s\le t\}$, augmented by the ${{\mathbb {P}}}$-null sets,

(c):

The system (2.2) has a unique solution $X(\cdot )$ with $X(t)\ge 0$ for any $t\ge 0$,

(d):

$0< J(x, Y, Z)<\infty$ for any $x\in S$, where $J(\cdot )$ is the functional defined in (2.4).

Remark 2.1

The condition $\Delta Y_i(t) \Delta Z_i(t)=0$, for any $i=1, \dots , d$ and any $t\ge 0$ means that we do not allow the simultaneous harvesting and seeding of the same species. Note that this would never yield an optimal harvesting-seeding strategy because of the assumption $f_i < g_i$.

The optimal harvesting-seeding problem The problem we will be interested in is to maximize the performance function and find an optimal harvesting strategy $(Y^*, Z^*)\in {\mathcal {A}}_{x}$ such that

$$\begin{aligned} J(x, Y^*, Z^*)=V(x):= \sup \limits _{(Y, Z)\in {\mathcal {A}}_{x}}J(x, Y, Z). \end{aligned}$$

(2.5)

The function $V(\cdot )$ is called the value function.

Remark 2.2

We note that the optimal harvesting strategy might not exist, i.e. the maximum over ${\mathcal {A}}_x$ might not be achieved in ${\mathcal {A}}_x$.

Assumption 2.3

We will make the following standing assumptions throughout the paper.

(a):

The functions $b(\cdot )$ and $\sigma (\cdot )$ are continuous. Moreover, for any initial condition $x\in {{\bar{S}}}$, the uncontrolled system (2.1) has a unique global solution.

(b):

For any $i=1, \dots , d$, $x, y\in \mathbb {R}^d$, ${f_i(x)}< g_i(y)$; $f_i(\cdot )$, $g_i(\cdot )$ are continuous and non-increasing functions.

Remark 2.4

Note that Assumption 2.3 (a) does not put significant restraints on the dynamics of the species. Our framework therefore contains a very broad class of models. In particular, this covers all Lotka-Volterra competition and predator–prey models as well as the more general Kolmogorov systems (Du et al. 2016; Li and Mao 2009; Mao and Yuan 2006; Hening and Nguyen 2018). The continuity and monotonicity of the functions $f(\cdot ), g(\cdot )$ from Assumption 2.3(b) are standard (Alvarez 2000; Song et al. 2011; Tran and Yin 2017). In particular, $g_i$ is non-increasing because one can see this as an economy of scale which implies that the per unit cost of restocking decreases as more is restocked The additional requirement that ${f_i(x)}< g_i(y)$ for any $x, y\in {{\bar{S}}}$ can be explained as follows: the cost of seeding an amount of a species is always higher than the benefit received from harvesting the same amount. This makes sense because in order to seed the species, one has to have access to a pool of individuals of this species. For this, one either has to keep individuals at a specific location (thus using resources to sustain them) or one has to transport/buy individuals. In the setting of optimal dividend payments, these extra costs reflect penalizing factors (Kulenko and Schmidli 2008) and transaction costs (Jin et al. 2013; Scheer and Schmidli 2011).

We collect some of the results we are able to prove about the value function.

Proposition 2.5

Let Assumption 2.3 be satisfied. Then the following assertions hold.

(a):

For any $x, y\in {{\bar{S}}}$,

$$\begin{aligned} V(y)\le V(x) - f(x)\cdot (x-y)^++g(x)\cdot (x-y)^-. \end{aligned}$$

(2.6)

(b):

If $V(0)<\infty$, then $V(x)<\infty$ for any $x\in {{\bar{S}}}$ and V is Lipschitz continuous.

Example 2.6

In the current setting, contrary to the regular harvesting setting without seeding, V(0) can be nonzero because of the benefits from seedings. Consider the single species system given by

$$\begin{aligned} dX(t)=X(t)(a-bX(t))dt + \sigma X(t) dw(t)-dY(t) +dZ(t), \quad X(0)=x,\nonumber \\ \end{aligned}$$

(2.7)

where a, b,  and $\sigma$ are constants and the price function is $f(x)=1, x\ge 0$. It has been shown in Alvarez and Shepp (1998) that, if there is no seeding, the value function is given by

$$\begin{aligned} V_0(x)= \psi (x) \text { for }x<x^*, \quad V_0(x)= x-x^*+\psi (x^*) \text { for }x\ge x^*, \end{aligned}$$

where $x^*\in (0, \infty )$ and $\psi : [0, \infty )\rightarrow [0, \infty )$ is twice continuously differentiable, and $\psi (x)>x$ for all $x\in (0, x^*]$. Let $g(x)=\kappa \in {\mathbb {R}}$, where $1<\kappa <\psi (x^*)/x^*$. Let $(Y, Z)\in {\mathcal {A}}_0$ be such that $J(0, Y, 0)=V_0(x)$ and $Z(t)=Z(0)=x^*$ for all $t\ge 0$. Then

$$\begin{aligned} V(0)\ge J(0, Y,Z)\ge \psi (x^*)-\kappa x^*>0. \end{aligned}$$

Since $V(0)>0$, the system does not get depleted in a finite time under an optimal harvesting strategy.

Proposition 2.7

Let Assumption 2.3 be satisfied. Moreover, suppose that there is a positive constant C such that

$$\begin{aligned} b_i(x)\le \delta x_i+C, \quad x\in {{\bar{S}}}, \quad i=1, \dots , d. \end{aligned}$$

(2.8)

Then there exist a positive constant M such that

$$\begin{aligned} V(x)\le \sum \limits _{i=1}^d f_i(0) x_i+ M, \quad x\in {{\bar{S}}}. \end{aligned}$$

Remark 2.8

We note that the condition on the drift $b(\cdot )$ is very natural. Consider the one-dimensional dynamics given by

$$\begin{aligned} dX(t)=bX(t)dt + \sigma X(t) dw(t)-dY(t) +dZ(t), \quad X(0)=x, \end{aligned}$$

(2.9)

with $f(x)=1, x\ge 0$ and any function $g(\cdot )$. It is clear that if $b>\delta$, the value function in the harvesting problem with no seeding is

$$\begin{aligned} V_0(x)=\inf \limits _{(Y, Z)\in {\mathcal {A}}_{x}, Z=0 }J(x, Y, Z)=\infty \quad \text {for all}\quad x>0. \end{aligned}$$

As a result the value function for (2.9) will be $V(x)=\infty , x>0.$

Seeding can also change the finiteness of the value function. Indeed, consider the harvesting problem

$$\begin{aligned} dX(t)=b(X(t))dt + \sigma (X(t)) dw(t)-dY(t) +dZ(t), \quad X(0)=x, \end{aligned}$$

(2.10)

with $f(x)=1, g(x)=2, x\ge 0$. Suppose that $g(x)=\sigma (x)=0$ for $x<1$ and $b(x)=(1+\delta ) x( 1-x)$ and $\sigma (x)=0$ for $x>1$. Then it is clear that without seeding we get the value function $V_0(x)=\infty$ for $x>1$ and $V_0(x)=x$ for $x\le 1$. When seeding is allowed, we have $V(x)=\infty$ for all $x\ge 0$.

We get the following characterization of the value function.

Theorem 2.9

Let Assumption 2.3 be satisfied and suppose $V(x)<\infty$ for $x\in {S}$. The value function $V(\cdot )$ is a viscosity solution to the HJB equation

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )V(x), f_i(x)-\dfrac{\partial V}{\partial x_i}(x), \dfrac{\partial V}{\partial x_i}(x)-g_i(x)\bigg \}= 0. \end{aligned}$$

(2.11)

Remark 2.10

Theorem 2.9 is a theorem that tells us how to find the value function. The problem is that the solutions of (2.11) are not always smooth enough for ${\mathcal {L}}V$ to make sense. This is why we work with viscosity solutions of (2.11).

We next explain what a viscosity solution means. For any $x^0\in S$ and any function $\phi \in C^2(S)$ such that $V(x_0)=\phi (x_0)$ and $V(x)\ge \phi (x)$ for all x in a neighborhood of $x^0$, we have

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\phi (x^0), f_i(x^0)-\dfrac{\partial \phi }{\partial x_i}(x^0), \dfrac{\partial \phi }{\partial x_i}(x^0)-g_i(x^0)\bigg \}\le 0. \end{aligned}$$

Similarly, for any $x^0\in S$ and any function $\varphi \in C^2(S)$ satisfying $V(x_0)=\phi (x_0)$ and $V(x)\le \phi (x)$ for all x in a neighborhood of $x^0$, we have

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\varphi (x^0), f_i(x^0)-\dfrac{\partial \varphi }{\partial x_i}(x^0), \dfrac{\partial \varphi }{\partial x_i}(x^0)-g_i(x^0)\bigg \}\ge 0. \end{aligned}$$

This extends the results by Haussmann and Suo (1995a, b), Lungu and Øksendal (2001) where the authors had to assume that the coefficients $b, \sigma$ are bounded or the prices $f_i$ are not density-dependent. Usually the functions $b,\sigma$ are not bounded and the prices depend on the abundances of the species. Moreover, we consider both harvesting and seeding. Therefore, our results provide a significant generalization of those previously shown by Haussmann and Suo (1995a), Lungu and Øksendal (2001).

We also get the following verification theorem, that tells us that if a function satisfies certain properties, then it will be the value function. We note that this is natural analogue with seeding of Theorem 2.1 of Lungu and Øksendal (2001), Alvarez et al. (2016).

Theorem 2.11

Let Assumption 2.3 be satisfied. Suppose that there exists a function $\Phi : {{\bar{S}}}\mapsto [0, \infty )$ such that $\Phi \in C^2({{\bar{S}}})$ and that $\Phi (\cdot )$ solves the following coupled system of quasi-variational inequalities

$$\begin{aligned} \sup \limits _{(x, i)} \bigg \{({\mathcal {L}}-\delta )\Phi (x), f_i(x)-\dfrac{\partial \Phi }{\partial x_i}(x), \dfrac{\partial \Phi }{\partial x_i}(x)-g_i(x)\bigg \}\le 0, \end{aligned}$$

(2.12)

where $({\mathcal {L}}-\delta )\Phi (x)={\mathcal {L}}\Phi (x)-\delta \Phi (x)$. Then the following assertions hold.

(a):

We have

$$\begin{aligned} V(x)\le \Phi (x)\quad \text {for any } x\in {S}. \end{aligned}$$

(2.13)

(b):

Define the non-intervention region

$$\begin{aligned} {\mathcal {C}}= \left\{ x\in S: f_i(x)<\dfrac{\partial \Phi }{\partial x_i}(x)<g_i(x) \right\} . \end{aligned}$$

Suppose that

$$\begin{aligned} ({\mathcal {L}}-r)\Phi (x)=0, \end{aligned}$$

(2.14)

for all $x\in {\mathcal {C}}$, and that there exists a harvesting-seeding strategy $\big (\widetilde{Y}, \widetilde{Z}\big ) \in {\mathcal {A}}_{x}$ and a corresponding process $\widetilde{X}$ such that the following statements hold.

(i):

$\widetilde{X}(t)\in \overline{{\mathcal {C}}}$ for Lebesgue almost all $t\ge 0.$

(ii):

$\int \nolimits _0^{t} \left[ \nabla \Phi (\widetilde{X}(s))- f(\widetilde{X}(s))\right] \cdot d\widetilde{Y}^c(s)=0$ for any $t\ge 0$.

(iii):

$\int \nolimits _0^{t} \Big [g(\widetilde{X}(s))-\nabla \Phi (\widetilde{X}(s)) \Big ] \cdot d\widetilde{Z}^c(s)=0$ for any $t\ge 0$.

(iv):

If $\widetilde{X}(s)\ne \widetilde{X}(s-)$, then

$$\begin{aligned} \Phi (\widetilde{X}(s))-\Phi (\widetilde{X}(s-))=- f(\widetilde{X}(s-)) \cdot \Delta \widetilde{Z}(s). \end{aligned}$$

(v):

$\lim \limits _{N\rightarrow \infty }E_{x}\Big [e^{-rT_N}\Phi (\widetilde{X}(T_N))\Big ]=0$, where for each $N=1, 2, \dots$,

$$\begin{aligned} \beta _N:=\inf \{t\ge 0: |X(t)|\ge N\},\quad T_N: = N \wedge \beta _N.\end{aligned}$$

(2.15)

Then $V(x)=\Phi (x)$ for all $x\in S$, and $\big (\widetilde{Y},\widetilde{Z}\big )$ is an optimal harvesting-seeding strategy.

Remark 2.12

Following (Lungu and Øksendal 2001) we note that if we can find a function satisfying (2.12), (2.13) and (2.14), then one can construct a strategy satisfying assumptions (i), (ii), (iii) and (iv) from Theorem 2.11 part b) by solving the Skorokhod stochastic differential equation for the reflection of the process X(t) in the domain ${\mathcal {C}}$. We refer the reader to check further literature (Lungu and Øksendal 2001; Bass 1998; Freidlin 2016; Lions and Sznitman 1984) for more details about Skorokhod stochastic differential equations.

We can extend Principle 2.1 of Lungu and Øksendal (2001) as follows.

Principle 2.13

(One-at-a-time principle) Suppose the diffusion matrix $\sigma (x)\sigma '(x)$ is nondegenerate for all $x\in S$. Then it is almost always optimal to harvest or to seed from at most one species at a time.

Proof

We follow (Lungu and Øksendal 2001). Assume for simplicity $d=2$ so that we have two species. The non-intervention region ${\mathcal {C}}$ is bounded by the four curves curves $\Lambda ^f_1, \Lambda ^f_2, \Lambda ^g_1, \Lambda ^g_2$ given by

$$\begin{aligned} \Lambda ^f_i = \left\{ (x_1,x_2)\in S~\Bigg |~ \frac{\partial \Phi }{\partial x_i}=f_i(x_1,x_2)\right\} \end{aligned}$$

and

$$\begin{aligned} \Lambda ^g_i = \left\{ (x_1,x_2)\in S~\Bigg |~ \frac{\partial \Phi }{\partial x_i}=g_i(x_1,x_2)\right\} . \end{aligned}$$

Note that we would have simultaneous harvesting and seeding of species i only when the process is at $\Lambda ^f_i \cap \Lambda ^g_i$, simultaneous harvesting of the two species only when the process is at $\Lambda ^f_1 \cap \Lambda ^f_2$, simultaneous harvesting of species 1 and seeding of species 2 only when the process is at $\Lambda ^f_1 \cap \Lambda ^g_2$, etc. Now, if the diffusion is non-degenerate, the probability it hits a set of the form $\Lambda ^f_i \cup \Lambda ^f_j$ for $i\ne j$ or $\Lambda ^f_i \cup \Lambda ^g_j$ is equal to zero. This argument can be extended to n dimensions—see Principle 2.1 of Lungu and Øksendal (2001). $\square$

2.1 Numerical scheme

A closed-form solution to the HJB equation from Theorem 2.9 is virtually impossible to obtain. Moreover, the initial value of V(0) is not specified. In order to by-pass these difficulties and to gain information about the value function and the optimal harvesting-seeding strategy we provide a numerical approach. Using the Markov chain approximation method (Budhiraja and Ross 2007; Jin et al. 2013; Kushner and Martins 1991; Kushner and Dupuis 1992), we construct a controlled Markov chain in discrete time to approximate the controlled diffusions. For the convergence analysis, we also suppose that both $f(\cdot )$ and $g(\cdot )$ are constant functions. Let $h>0$ be a discretization parameter. Since the real population abundances cannot be infinite, we choose a large number $U>0$ and define the class ${\mathcal {A}}^U_{x}\subset {\mathcal {A}}_{x}$ that consists of strategies $(Y,Z)\in {\mathcal {A}}_x$ such that the resulting process X stays in $[0,U]^d$ for all times. The class ${\mathcal {A}}^U_{x}$ can be constructed by using Skorokhod stochastic differential equations (Bass 1998; Freidlin 2016; Lions and Sznitman 1984) in order to make sure that the process stays in $[0,U]^d$ for all $t>0$.

Let $({\tilde{Y}}^U, {\tilde{Z}}^U)\in {\mathcal {A}}^U_{x}$ and $V^U(x)$ be defined as the optimal harvesting-seeding strategy and the value function when we restrict the problem to the class ${\mathcal {A}}^U_{x}\subset {\mathcal {A}}_{x}$

$$\begin{aligned} J(x, {\tilde{Y}}^U, {\tilde{Z}}^U)=V^U(x):=\sup \limits _{(Y, Z)\in {\mathcal {A}}_{x}^U} J(x, Y, Z) \end{aligned}$$

(2.16)

Remark 2.14

We conjecture that, generically, the optimal strategy will live in ${\mathcal {A}}_x^U$ for U large enough, i.e. there exists $U>0$ such that for all $x\in [0,U]^d$ we have

$$\begin{aligned} J(x, Y^*, Z^*)=V(x):= & {} \sup \limits _{(Y, Z)\in {\mathcal {A}}_{x}}J(x, Y, Z) =V^U(x)\\:= & {} \sup \limits _{(Y, Z)\in {\mathcal {A}}_{x}^U} J(x, Y, Z)=J(x, {\tilde{Y}}^U, {\tilde{Z}}^U). \end{aligned}$$

The verification Theorem 2.11 provides a heuristic argument for this conjecture.

Assume without loss of generality that U is an integer multiple of h. Define

$$\begin{aligned} S_{h}: = \{x=(k_1 h, \dots , k_d h)'\in \mathbb {R}^d: k_i=0, 1, 2, \dots \}\cap [0, U]^d. \end{aligned}$$

Let $\{X^h_n: n=0, 1, \dots \}$ be a discrete-time controlled Markov chain with state space $S_{h}$. We define the difference

$$\begin{aligned} \Delta X_n^h = X_{n+1}^h-X_{n}^h. \end{aligned}$$

At any discrete-time step n, one can either harvest, seed, or do nothing. We use $\pi ^h_n$ to denote the action at step n, where $\pi ^h_n=-i$ if there is seeding of species i, $\pi ^h_n=0$ if there is no seeding or harvesting of species i, and $\pi ^h_n=i$ if there is harvesting. Denote by $\Delta Y^h_n$ and $\Delta Z^h_n$ the harvesting amount and the seeding amount for the chain at step n, respectively. If $\pi ^h_n=0$, then the increment $\Delta X_n^h$ is to behave like an increment of $\int b dt +\int \sigma dw$ over a small time interval. Such a step is also called “diffusion step”. If $\pi ^h_n=-i$, then $\Delta Y^h_n=0\in \mathbb {R}^d$ and $\Delta Z^h_n=h\mathbf{e_i}$. If $\pi ^h_n=i$, then $\Delta Y^h_n=h\mathbf{e_i}$ and $\Delta Z^h_n=0\in \mathbb {R}^d$. Note that $\Delta X^h_n=-\Delta Y^h_n + \Delta Z^h_n$. Moreover, we can write

$$\begin{aligned} \Delta X_n^h= & {} \Delta X_n^h I_{\{ \text {diffusion step at } n\}} + \Delta X_n^h I_{\{\text {harvesting step at }n\}}\nonumber \\&+ \Delta X_n^h I_{\{\text {seeding step at }n\}}. \end{aligned}$$

(2.17)

For definiteness, if $X^{h}_{n, i}$ is the ith component of the vector $X^h_n$ and $\{j: X^{h}_{n, j}=U\}$ is non-empty, then step n is a harvesting step on species $\min \{j: X_{n, j}^{h}=U\}$. Let $\pi ^h = (\pi _0^h, \pi _1^h, \dots )$ denote the sequence of control actions. We denote by $p^h(x, y)|\pi )$ the transition probability from state x to another state y under the control $\pi$. Denote ${\mathcal {F}}^h_n=\sigma \{X^h_m, \pi ^h_m, m\le n\}$.

The sequence $\pi ^h$ is said to be admissible if it satisfies the following conditions:

(a):

$\pi ^h_n$ is $\sigma \{X^h_0, \dots , X^h_{n}, \pi ^h_0, ..., \pi ^h_{n-1}\}\text {---adapted},$

(b):

For any $x\in S_h$, we have

$$\begin{aligned} {{\mathbb {P}}}\{ X^h_{n+1} =x | {\mathcal {F}}^h_n\}= {{\mathbb {P}}}\{X^h_{n+1}= x | X^h_n, \pi ^h_n\} = p^h( X^h_n, x| \pi ^h_n), \end{aligned}$$

(c):

Denote by $X^{h}_{n, i}$ the ith component of the vector $X^h_n$. Then

$$\begin{aligned} {{\mathbb {P}}}\big ( \pi ^h_{n}=\min \{j: X^{h}_{n, j} = U\} | X^{h}_{n, j} = U \text { for some } j\in \{1, \dots , d \}, {\mathcal {F}}^h_n\big )=1.\nonumber \\ \end{aligned}$$

(2.18)

(d):

$X^h_n\in S_h$ for all $n=0, 1, 2, \dots$.

The class of all admissible control sequences $\pi ^h$ for initial state x will be denoted by ${\mathcal {A}}^h_{x}$.

For each couple $(x, i)\in S_h\times \{0, \pm 1, \dots , \pm d\}$, we define a family of the interpolation intervals $\Delta t^h (x, i)$. The values of $\Delta t^h (x, i)$ will be specified later. Then we define

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle t^h_0 = 0,\quad \Delta t^h_m = \Delta t^h(X^h_m, \pi ^h_m), \quad t^h_n = \sum \limits _{m=0}^{n-1} \Delta t^h_m.\\ \end{array} \end{aligned}$$

(2.19)

For $x\in S_h$ and $\pi ^h\in {\mathcal {A}}^h_{x}$, the performance function for the controlled Markov chain is defined as

$$\begin{aligned} J^h(x, \pi ^h) = {{\mathbb {E}}}\sum _{m=1}^{\infty } e^{-\delta t_m^h}\bigg [f (X^h_m)\cdot \Delta Y_{m}^{h}-g(X^h_m) \cdot \Delta Z_{m}^{h} \bigg ]. \end{aligned}$$

(2.20)

The value function of the controlled Markov chain is

$$\begin{aligned} V^h(x) = \sup \limits _{\pi ^h\in {\mathcal {A}}^h_{x}} J^h (x, \pi ^h). \end{aligned}$$

(2.21)

Theorem 2.15

Suppose Assumptions 2.3 and B.1 hold. Then $V^h(x)\rightarrow V^U(x)$ as $h\rightarrow 0$. Thus, for sufficiently small h, a near-optimal harvesting-seeding strategy of the controlled Markov chain is also a near-optimal harvesting-seeding policy of the original continuous-time problem.

3 Numerical examples

3.1 Single species system

We consider a single species ecosystem. The dynamics that includes harvesting and seeding will be given by

$$\begin{aligned} d X(t) = X(t)\big (b-cX(t)\big ) + \sigma X(t) dw(t)-dY(t) +dZ(t). \end{aligned}$$

(3.1)

For an admissible strategy (Y, Z) we have

$$\begin{aligned} J(x, Y, Z)={{\mathbb {E}}}\left[ \int _0^\infty e^{-\delta s} f(X(s-))dY(s)-\int _0^\infty e^{-\delta s} g(X(s-))dZ(s)\right] .\qquad \end{aligned}$$

(3.2)

Based on the algorithm constructed above and in Appendix B, we carry out the computation by value iterations. Let $(Y_0,Z_0)$ be the policy that drives the system to extinction immediately and has no seeding. Then $J(x, Y_0, Z_0)=f(x)x$ for all x. Recall (Alvarez and Shepp 1998) that $J(x, Y_0, Z_0)$ is also referred to as current harvesting potential. Letting $(Y_0, Z_0)$ be the initial strategy, we set the initial values

$$\begin{aligned} V_0^h(x)=f(x)x, \quad x = 0, h, 2h, \dots , U = 10. \end{aligned}$$

We outline how to find the values of $V(\cdot )$ as follows. At each level $x=h,2h, \dots , U$, denote by $\pi (x, n)$ the action one chooses, where $\pi (x, n)=1$ if there is harvesting, $\pi (x, n)=-1$ if there is seeding, and $\pi (x, n)=0$ if there is no harvesting or seeding. We initially let $\pi (x, 0)=1$ for all x and we try to find better harvesting-seeding strategies. We find an improved value $V^h_{n+1}(x)$ and record the corresponding optimal action by

$$\begin{aligned} \pi (x, n)=\hbox {argmax}\left\{ i=-1, 0, 1: V^{h, i}_{n+1} (x, \alpha )\right\} ,\quad V^h_{n+1}(x)= V^{h, \pi (x, n)}_{n+1} (x), \end{aligned}$$

where

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle V^{h, 1}_{n+1}(x)=V^h_n(x-h) + f(x)h,\\ &{}\displaystyle V^{h, -1}_{n+1}(x)=V^h_n(x+h) - g (x)h,\\ &{}\displaystyle V^{h}_{n+1, 0} (x) =e^{-\delta \Delta t^h(x, 0) }\Big [ V^{h}_{n} (x+h) p^h (x, x+h | \pi )+V^{h}_{n} (x-h) p^h ( x, x-h | \pi \big ) \Big ]. \end{array} \end{aligned}$$

The iterations stop as soon as the increment $V^h_{n+1}(\cdot )-V^h_n(\cdot )$ reaches some tolerance level. We set the error tolerance to be $10^{-8}$.

The numerical experiments provide evidence that the following conjecture holds

Conjecture 3.1

Suppose we have one species that evolves according to (3.1) and suppose Assumption 2.3 holds. One can construct the optimal harvesting-seeding strategy $(Y^*,Z^*)$ as follows. There exist lower and upper thresholds $0\le u^*< v^*<\infty$ such that after $t\ge 0$ the abundance of the species always stays in the interval $[u^*,v^*]$. More explicitly if $X(0-)=x$ then

$$\begin{aligned} \begin{aligned} (Y^{u^*}(t), Z^{v^*}(t))&= {\left\{ \begin{array}{ll} \left( (x-v^*)^+,(u^*-x)^+\right) &{}\quad {if t=0,} \\ (L(t,v^*), L(t,u^*)) &{} \quad {if t>0} \end{array}\right. } \end{aligned} \end{aligned}$$

(3.3)

where $L(t,u^*)$ (respectively $L(t,v^*)$) is the local time push of the process X at the boundary $u^*$ (respectively $v^*$).

Fig. 1

Value function and optimal policies: $f(x)=1, g(x)=3$ for $x\ge 0.$

For the first numerical experiment we take $b=3, c=2, \sigma =1$ in (3.1). Let $\delta =0.05$, and $f(x)=1,g(x)=3$ for all $x\ge 0$. Figure 1 shows the value function V(x) as a function of the initial population x and provides optimal policies, with 1 denoting harvesting, $-1$ denoting seeding, and 0 denoting no harvesting or seeding. It can be seen from Fig. 1 that the optimal policy is a barrier strategy. There are levels $L_1$ and $L_2$ such that $[0, L_1)$ is the seeding region, $[L_1, L_2)$ is the diffusion region where there is no control of the population, and $[L_2, U]$ is the harvesting region. Because of the benefit from seeding, $V(0)>0$. These observations agree with those in the analogous financial setting (Jin et al. 2013; Scheer and Schmidli 2011).

Next, suppose that g takes very large values. In particular, we take $g(x)=50, x\ge 0$. In this case, one can observe that there is no seeding; see Fig. 2. In other words, because the cost of seeding is very high, the optimal strategy does not benefit from seeding.

To explore how noise impacts the problem, we explore what happens when $\sigma =1000$ and keep the other coefficients the same. The results are shown on Fig. 3. It turns out, as expected, that if the noise is very large, the value function is close to the current harvesting potential $J(\cdot , Y_0, Z_0)$ and no seeding is needed. This is because the large noise will drive the species extinct with probability 1 and, therefore, the optimal strategy is to immediately harvest all individuals. We refer the reader to the works by Alvarez and Shepp (1998), Tran and Yin (2016), Hening et al. (2019), Alvarez and Hening (2018) for more insight regarding how noise impacts harvesting.

Fig. 2

Value function and optimal policies: $f(x)=1, g(x)=50$ for $x\ge 0$

Fig. 3

Value function and optimal policies: $f(x)=1, g(x)=3, \sigma (x)=1000$ for $x\ge 0.$

Fig. 4

Value function versus initial population for a two-species competitive model

3.2 Two-species ecosystems

Example 3.2

Consider two species competing according to the following stochastic Lotka-Volterra system

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle d X_1(t) = X_1(t)\Big (b_1-a_{11}X_1(t) - a_{12} X_2(t) \Big ) + \sigma _1X_1(t)dw(t)-dY_1(t) + dZ_1(t)\\ &{}\displaystyle d X_2(t) = X_2(t)\Big (b_2-a_{21}X_1(t) - a_{22} X_2(t) \Big ) + \sigma _2X_2(t)dw(t)-dY_2(t) + dZ_2(t) \end{array}\nonumber \\ \end{aligned}$$

(3.4)

and suppose that $\delta =0.05$, $f_1(x)=1$, $f_2(x)=2$, $g_1(x)=4, g_2(x)=4$ for all $x\in [0, \infty )^2$. We take $U = 5$. In addition, set

$$\begin{aligned} b_1=3, a_{11}=2, a_{12}=1, \sigma _1=3, b_2=2, a_{21}=1, a_{22}=2, \sigma _2=3. \end{aligned}$$

Figure 4 shows the value function V as a function of initial population sizes (x, y). Figure 5 provides the optimal harvesting-seeding policies, with “1” denoting harvesting of species 1, “$-1$” denoting seeding of species 1, “2” denoting harvesting of species 2, “$-2$” denoting seeding of species 2 and “0” denoting no harvesting or seeding. It can be seen from Fig. 5 that when population size of each species is larger than a certain level, it is optimal to harvest. However, for a very large region, harvesting of species 1 is the first choice. Moreover, one can observe that it is never optimal to seed species 1 (Fig. 6). As shown in Fig. 7, if we assume both species abundances are 0 initially, we should only seed species 2. This tells us that the benefits obtained from species 2 are larger than those from species 1 due to its higher price; i.e, $f_2(x)=2>1=f_1(x)$.

Fig. 5

Optimal harvesting-seeding strategy versus population size for a two-species competitive model

Fig. 6

Optimal harvesting-seeding strategy versus population size for a two-species competitive model: the case $y=0$ (the left picture) and $x=0$ (the right picture)

Fig. 7

Value function versus initial population for a two-species predator–prey model

Fig. 8

Optimal harvesting-seeding strategy versus population size for a two-species predator–prey model

Example 3.3

Consider a predator–prey system modelled by the stochastic Lotka–Volterra system

$$\begin{aligned} d X_1(t)= & {} X_1(t)\Big (b_1-a_{11}X_1(t) - a_{12} X_2(t) \Big )\nonumber \\&+\, \sigma _1X_1(t)dw(t)-dY_1(t) + dZ_1(t)\nonumber \\ d X_2(t)= & {} X_2(t)\Big (\!-b_2 + a_{21}X_1(t) - a_{22} X_2(t) \Big )\nonumber \\&+\, \sigma _2X_2(t)dw(t)-dY_2(t) + dZ_2(t). \end{aligned}$$

(3.5)

Conditions for the coexistence and extinction of the different species can be found in Hening and Nguyen (2018). Suppose that $\delta =0.05$, $f_1(x)=1$, $f_2(x)=1$, $g_1(x)=6, g_2(x)=6$ for all $x\in [0, \infty )^2$. We take $U = 5$. In addition, let

$$\begin{aligned} b_1=2, a_{11}=1.2, a_{12}=1, \sigma _1=1.2, b_2=1, a_{21}=1.2, a_{22}=7, \sigma _2=1.3. \end{aligned}$$

Fig. 9

Optimal harvesting-seeding strategy versus population size for a two-species predator–prey model: the case $y=0$ (the left picture) and $x=0$ (the right picture)

Figure 7 shows the value function V as a function of initial population (x, y). Figure 8 provides the optimal harvesting-seeding strategies, with “1” denoting harvesting on species 1, “$-1$” denoting seeding on species 1, “2” denoting harvesting on species 2, “$-2$” denoting seeding on species 2 and “0” denoting no harvesting or seeding. We see in Fig. 9 that, as expected, since the predator goes extinct if there is no prey, the optimal strategy is to immediately harvest all the predators at time $t=0$.

Appendices

Appendix A: Properties of value functions

This section is devoted to several properties of the value function. Particularly, the lemma below will be helpful in proving the boundedness of the value function.

Lemma A.1

Suppose that Assumption 2.3 holds, (Y, Z) is an admissible control policy yielding the process X, and $\Phi (\cdot )\in C^2(\mathbb {R}^d)$. Then for any $s\ge 0$, there exist ${\widehat{X}}(s)\in \mathbb {R}^d$ and ${\widetilde{X}}(s)\in \mathbb {R}^d$ such that ${\widehat{X}}(s)\le X(s)$, ${\widetilde{X}}(s)\le X(s)$, and

$$\begin{aligned} \Phi ({X}(s) )- \Phi ({X}(s-))= -\Delta Y(s)\cdot \nabla \Phi ({\widehat{X}}(s)) + \Delta Z(s)\cdot \nabla \Phi ({\widetilde{X}}(s)). \end{aligned}$$

Proof

Without loss of generality, we suppose that $\Delta Y_i(s)> 0$ for $i=1, \dots , k$ and $\Delta Y_i(s)=0$ for $i=k+1, \dots , {d}$, where $k\le d$. Define

$$\begin{aligned} {X}^*(s)=\big (X_1(s-), \dots , X_k(s-), X_{k+1}(s), \dots , X_d(s)\big )'. \end{aligned}$$

By the admissibility of (Y, Z), $\Delta Z_i(s)= 0$ for $i=1, \dots , k$ and $\Delta Y_i(s)=0$ for $i={k+1}, \dots , d$. We can check that

$$\begin{aligned} X(s)-{X}^*(s)=-\Delta Y(s)\le 0 \quad \text {and} \quad {X}^*(s)-{X}(s-)=\Delta Z(s) \ge 0. \end{aligned}$$

By the mean value theorem, there is a point ${\widehat{X}}(s)\le X(s)$ on the line segment connecting X(s) and ${X}^*(s)$ such that

$$\begin{aligned} \Phi ({X}(s) )- \Phi ({X}^*(s)) = -\Delta Y(s)\cdot \nabla \Phi ({\widehat{X}}(s)). \end{aligned}$$

(A.1)

Similarly, there is a point ${\widetilde{X}}(s)\le X(s)$ on the line segment connecting ${X}(s-)$ and ${X}^*(s)$ such that

$$\begin{aligned} \Phi ({X}^*(s) )- \Phi ({X}(s-)) = \Delta Z(s)\cdot \nabla \Phi ({\widetilde{X}}(s)). \end{aligned}$$

(A.2)

The conclusion follows from (A.1) and (A.2). $\square$

Theorem 2.11

Let Assumption 2.3 be satisfied. Suppose that there exists a function $\Phi : {{\bar{S}}}\mapsto [0, \infty )$ such that $\Phi \in C^2({{\bar{S}}})$ and that $\Phi (\cdot )$ solves the following coupled system of quasi-variational inequalities

$$\begin{aligned} \sup \limits _{(x, i)} \bigg \{({\mathcal {L}}-\delta )\Phi (x), f_i(x)-\dfrac{\partial \Phi }{\partial x_i}(x), \dfrac{\partial \Phi }{\partial x_i}(x)-g_i(x)\bigg \}\le 0, \end{aligned}$$

(2.12)

where $({\mathcal {L}}-\delta )\Phi (x)={\mathcal {L}}\Phi (x)-\delta \Phi (x)$. Then the following assertions hold.

(a):

We have

$$\begin{aligned} V(x)\le \Phi (x)\quad \text {for any } x\in {S}. \end{aligned}$$

(2.13)

(b):

Define the non-intervention region

$$\begin{aligned} {\mathcal {C}}= \left\{ x\in S: f_i(x)<\dfrac{\partial \Phi }{\partial x_i}(x)<g_i(x) \right\} . \end{aligned}$$

Suppose that

$$\begin{aligned} ({\mathcal {L}}-r)\Phi (x)=0, \end{aligned}$$

(2.14)

for all $x\in {\mathcal {C}}$, and that there exists a harvesting-seeding strategy $\big (\widetilde{Y}, \widetilde{Z}\big ) \in {\mathcal {A}}_{x}$ and a corresponding process $\widetilde{X}$ such that the following statements hold.

(i):

$\widetilde{X}(t)\in \overline{{\mathcal {C}}}$ for Lebesgue almost all $t\ge 0.$

(ii):

$\int \nolimits _0^{t} \left[ \nabla \Phi (\widetilde{X}(s))- f(\widetilde{X}(s))\right] \cdot d\widetilde{Y}^c(s)=0$ for any $t\ge 0$.

(iii):

$\int \nolimits _0^{t} \Big [g(\widetilde{X}(s))-\nabla \Phi (\widetilde{X}(s)) \Big ] \cdot d\widetilde{Z}^c(s)=0$ for any $t\ge 0$.

(iv):

If $\widetilde{X}(s)\ne \widetilde{X}(s-)$, then

$$\begin{aligned} \Phi (\widetilde{X}(s))-\Phi (\widetilde{X}(s-))=- f(\widetilde{X}(s-)) \cdot \Delta \widetilde{Z}(s). \end{aligned}$$

(v):

$\lim \limits _{N\rightarrow \infty }E_{x}\Big [e^{-rT_N}\Phi (\widetilde{X}(T_N))\Big ]=0$, where for each $N=1, 2, \dots$,

$$\begin{aligned} \beta _N:=\inf \{t\ge 0: |X(t)|\ge N\},\quad T_N: = N \wedge \beta _N. \end{aligned}$$

(2.15)

Then $V(x)=\Phi (x)$ for all $x\in S$, and $\big (\widetilde{Y},\widetilde{Z}\big )$ is an optimal harvesting-seeding strategy.

Proof

(a) Fix some $x \in S$ and $(Y, Z)\in {\mathcal {A}}_{x}$, and let X denote the corresponding harvested process. Choose N sufficiently large so that $|x|<N$. For

$$\begin{aligned} \beta _N=\inf \{t\ge 0: |X(t)|\ge N\}, \quad T_N = N\wedge \beta _N, \end{aligned}$$

by the admissibility of (Y, Z), we have

$$\begin{aligned} \beta _N\rightarrow \infty \quad \text {and}\quad T_N \rightarrow \infty \quad \text {almost surely as } \quad N \rightarrow \infty . \end{aligned}$$

(A.3)

Then Dynkin’s formula leads to

$$\begin{aligned} \begin{array}{ll} {{\mathbb {E}}}_{x}\, &{}\!\displaystyle \big [e^{-\delta T_N}\Phi (X(T_N))\big ]-\Phi (x)\\ &{}\!\displaystyle = {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}({\mathcal {L}}-\delta )\Phi (X(s))ds-{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (X(s))\cdot dY^c(s)\\ &{}\!\displaystyle \quad +{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (X(s))\cdot dZ^c(s)\\ &{}\!\displaystyle \quad + {{\mathbb {E}}}_{x}\sum \limits _{0\le s\le T_N}e^{-\delta s}\Big [\Phi (X(s))-\Phi (X(s-))\Big ]. \end{array} \end{aligned}$$

It follows from (2.12) that

$$\begin{aligned} \begin{array}{ll} {{\mathbb {E}}}_{x} &{}\!\displaystyle \big [e^{-\delta T_N}\Phi (X(T_N))\big ]-\Phi (x)\\ &{}\!\displaystyle \le -{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (X(s))\cdot dY^c(s) + {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (X(s))\cdot dZ^c(s)\\ &{}\!\displaystyle \quad + {{\mathbb {E}}}_{x}\sum \limits _{0\le s\le T_N}e^{-\delta s}\Delta \Phi (X(s)), \end{array} \end{aligned}$$

(A.4)

where $\Delta \Phi (X(s))=\Phi (X(s))-\Phi (X(s-))$. By virtue of Lemma A.1, the monotonicity of $f(\cdot )$, $g(\cdot )$, and (2.12), we obtain

$$\begin{aligned} \Delta \Phi (X(s))\le -f(X(s-))\cdot \Delta Y(s)) +g(X(s-))\cdot \Delta Z(s). \end{aligned}$$

(A.5)

Since $\Phi (\cdot )$ is nonnegative, it follows from (A.4) and (A.5) that

$$\begin{aligned} \begin{array}{ll} \Phi (x) &{}\!\displaystyle \ge \bigg [ {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}f(X(s-))\cdot dY^c(s) + {{\mathbb {E}}}_{x}\sum \limits _{0\le s\le T_N}e^{-\delta s}f(X(s-))\cdot \Delta Y(s)\bigg ] \\ &{}\!\displaystyle \quad - \bigg [ {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}g(X(s-))\cdot dZ^c(s) + {{\mathbb {E}}}_{x}\sum \limits _{0\le s\le T_N}e^{-\delta s}g(X(s-))\cdot \Delta Z(s)\bigg ] \\ &{}\!\displaystyle = {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}f(X(s-))\cdot dY(s)-{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}g(X(s-))\cdot dZ(s). \end{array} \end{aligned}$$

Letting $N\rightarrow \infty$, it follows from (A.3) and the bounded convergence theorem that

$$\begin{aligned} \Phi (x)\ge {{\mathbb {E}}}_{x}\bigg [\int _0^\infty e^{-\delta s}f(X(s-))\cdot dY(s)-\int _0^\infty e^{-\delta s}g(X(s-))\cdot dZ(s)\bigg ]. \end{aligned}$$

Taking supremum over all $(Y, Z)\in {\mathcal {A}}_{x}$, we obtain $\Phi (x)\ge V(x)$.

(b) Let (i)–(v) be satisfied. Then Dynkin’s formula leads to

$$\begin{aligned} \begin{array}{ll} {{\mathbb {E}}}_{x} &{}\!\displaystyle \big [e^{-\delta T_N}\Phi (\widetilde{X}(T_N))\big ]-\Phi (x)\\ &{}\!\displaystyle = {{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}({\mathcal {L}}-\delta )\Phi (\widetilde{X}(s))ds-{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (\widetilde{X}(s))\cdot d\widetilde{Y}^c(s)\\ &{}\!\displaystyle \quad +{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{-\delta s}\nabla \Phi (\widetilde{X}(s))\cdot d\widetilde{Z}^c(s) + {{\mathbb {E}}}_{x}\sum \limits _{0\le s\le T_N}e^{-\delta s}\Big [\Phi (\widetilde{X}(s))-\Phi (\widetilde{X}(s-))\Big ]. \end{array} \end{aligned}$$

By (i), $({\mathcal {L}}-\delta )\Phi (\widetilde{X}(s))=0$ for almost all $s\ge 0$. This, together with (ii) and (iv), implies that

$$\begin{aligned} \begin{array}{ll} \Phi (x)&{}\!\displaystyle \,={{\mathbb {E}}}_{x} \big [e^{-\delta T_N}\Phi \big (\widetilde{X}(T_N)\big )\big ]+{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}f(\widetilde{X}(s-))\cdot d\widetilde{Y}(s)\\ &{}\!\displaystyle \qquad -{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}g(\widetilde{X}(s-))\cdot d\widetilde{Z}(s). \end{array} \end{aligned}$$

Letting $N\rightarrow \infty$ and using (iv), we obtain

$$\begin{aligned} \Phi (x)={{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}f(\widetilde{X}(s-))\cdot d\widetilde{Y}(s) -{{\mathbb {E}}}_{x}\int _{0}^{T_N} e^{- \delta s}g(\widetilde{X}(s-))\cdot d\widetilde{Z}(s). \end{aligned}$$

This, together with (a), implies that $\Phi (x)=V(x)$ for any $x\in S$ and $(\widetilde{Y}, \widetilde{Z})$ is an optimal harvesting strategy. $\square$

Next, we establish the continuity of the value function.

Proposition 2.5

Let Assumption 2.3 be satisfied. Then the following assertions hold.

(a):

For any $x, y\in {{\bar{S}}}$,

$$\begin{aligned} V(y)\le V(x) - f(x)\cdot (x-y)^++g(x)\cdot (x-y)^-. \end{aligned}$$

(2.6)

(b):

If $V(0)<\infty$, then $V(x)<\infty$ for any $x\in {{\bar{S}}}$ and V is Lipschitz continuous.

Proof

(a) Fix $(Y, Z)\in {\mathcal {A}}_{y}$. Define

$$\begin{aligned} {{\widetilde{Y}}}(t)=Y(t)+(x-y)^+, \quad {{\widetilde{Z}}}(t)=Z(t) + (x-y)^-, \quad t\ge 0. \end{aligned}$$

Then $({\widetilde{Y}}, {\widetilde{Z}})\in {\mathcal {A}}_{x}$. Let ${\widehat{X}}$ denote the process satisfying (2.2) with ${\widehat{X}}(0-)=x$ and strategy (Y, Z). Let ${\widetilde{X}}$ denote the process satisfying (2.2) with ${\widetilde{X}}(0-)=y$ and strategy $({\widetilde{Y}}, {\widetilde{Z}})$. Then we have ${\widehat{X}}(t)={\widetilde{X}}(t)$ for any $t> 0$. Consequently, it follows that

$$\begin{aligned} J(x, {\widetilde{Y}}, {\widetilde{Z}})=f(x)\cdot (x-y)^+-g(x)\cdot (x-y)^- + J(y, Y, Z). \end{aligned}$$

Since $V(x)\ge J(x, {\widetilde{Y}}, {\widetilde{Z}})$, we have

$$\begin{aligned} V(x)\ge f(x)\cdot (x-y)^+-g(x)\cdot (x-y)^- + J(y, Y, Z), \end{aligned}$$

from which, (2.6) follows by taking supremum over $(Y, Z)\in {\mathcal {A}}_{y}$.

(b) Similar to (2.6), we have

$$\begin{aligned} V(x)\le V(y) - f(y)\cdot (y-x)^++g(y)\cdot (y-x)^-. \end{aligned}$$

(A.6)

In view of (2.6) and (A.6), if $V(0)<\infty$, then $V(x)<\infty$ for any $x\in {{\bar{S}}}$. Moreover,

$$\begin{aligned}\begin{array}{ll} |V(x)-V(y)|&{}\!\displaystyle \le |f(x) + f(y) + g(x)+g(y)| |x-y|\\ &{}\!\displaystyle \le 2|f(0)+g(0)| |x-y|, \quad x, y\in S. \end{array} \end{aligned}$$

Thus, $V(\cdot )$ is Lipschitz continuous. $\square$

Using Theorem 2.11, we proceed to present an easily verifiable condition for the finiteness of the value function.

Proposition 2.7

Let Assumption 2.3 be satisfied. Moreover, suppose that there is a positive constant C such that

$$\begin{aligned} b_i(x)\le \delta x_i+C, \quad x\in {{\bar{S}}}, \quad i=1, \dots , d. \end{aligned}$$

(2.8)

Then there exist a positive constant M such that

$$\begin{aligned} V(x)\le \sum \limits _{i=1}^d f_i(0) x_i+ M, \quad x\in {{\bar{S}}}. \end{aligned}$$

Proof

Define

$$\begin{aligned} \Phi (x)=\sum \limits _{i=1}^d f_i(0) x_i + M, \quad x\in {{\bar{S}}}, \end{aligned}$$

where M is a positive number to be specified. We can check that $\Phi (\cdot )$ solves the system of inequalities (2.12) for sufficiently large M. By virtue of Theorem 2.11, $V(x)\le \Phi (x)$ for any $x \in S.$ The details are omitted for brevity. $\square$

Throughout the rest of this section, we aim to characterize the value function as a viscosity solution of an associated system of quasi-variational inequalities. Our approach is motivated by Asmussen and Taksar (1997), Song et al. (2011). However, the results and proofs below are nontrivial extensions because we have interacting species as well as seeding. We use the following notation and definitions. For a point $x^0\in S$ and a strategy $(Y, Z)\in {\mathcal {A}}_{x^0}$, let X be the corresponding process with harvesting and seeding. Let $B_\varepsilon (x^0)=\{x\in S: |x-x^0|<\varepsilon \}$, where $\varepsilon >0$ is sufficiently small so that $\overline{B_\varepsilon (x^0)}\subset S$. Let $\theta =\inf \{t\ge 0: {X}(t)\notin B_\varepsilon (x^0) \}$. For a constant $r>0$, we define $\theta _r=\theta \wedge r$.

Proposition A.2

Let Assumption 2.3 be satisfied and suppose that $V(x)<\infty$ for all $x\in {{\bar{S}}}$. The value function V is a viscosity subsolution of the system of quasi-variational inequalities

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\Phi (x), f_i(x)-\dfrac{\partial \Phi }{\partial x_i}(x), \dfrac{\partial \Phi }{\partial x_i}(x)-g_i(x)\bigg \}=0, \quad x\in S. \end{aligned}$$

(A.7)

That is, for any $x^0\in S$ and any function $\phi \in C^2(S)$ satisfying

$$\begin{aligned} (V-\phi )(x)\ge (V-\phi )(x^0)=0, \end{aligned}$$

for all x in a neighborhood of $x^0$, we have

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\phi (x^0), f_i(x^0)-\dfrac{\partial \phi }{\partial x_i}(x^0), \dfrac{\partial \phi }{\partial x_i}(x^0)-g_i(x^0)\bigg \}\le 0. \end{aligned}$$

(A.8)

Proof

For $x^0\in S$, consider a $C^2$ function $\phi (\cdot )$ satisfying $\phi (x^0)=V(x^0)$ and $\phi (x)\le V(x)$ for all x in a neighborhood of $x^0$. Let $\varepsilon >0$ be sufficiently small so that $\overline{B_\varepsilon (x^0)}\subset S$ and $\phi (x)\le V(x)$ for all $x\in \overline{B_\varepsilon (x_0)}$, where $\overline{B_\varepsilon (x_0)}=\{x\in S: |x-x^0|\le \varepsilon \}$ is the closure of $B_\varepsilon (x^0)$.

Choose $(Y, Z)\in {\mathcal {A}}_{x^0}$ such that $Y(0-)=Z(0-)=0$, $Y(t)=Y(0)$ and $Z(t)=Z(0)$ for any $t\ge 0$, $|\Delta Y(0)|+|\Delta Z(0)|\le \eta$, where $\eta \in [0, \varepsilon )$. Thus, there are only jumps at time $t=0$. Let X be the corresponding harvested process with initial condition $x^0$ and strategy (Y, Z).

Note that the chosen strategy (Y, Z) guarantees that X has at most one jump at $t=0$ and remains continuous on $(0, \infty )$. This, together with the fact that $\eta \in [0, \varepsilon )$, implies that ${X}(t)\in \overline{B_\varepsilon (x^0)}$ for all $0\le t\le \theta$. By virtue of the dynamic programming principle, we have

$$\begin{aligned} \phi (x^0)= & {} V(x^0)\nonumber \\\ge & {} {{\mathbb {E}}}\bigg [ \int _0^{\theta _r} e^{-\delta s}f({X}(s-))\cdot dY(s) - \int _0^{\theta _r} e^{-\delta s}g({X}(s-))\cdot dZ(s) + e^{-\delta \theta _r} V({X}(\theta _r))\bigg ]\nonumber \\\ge & {} {{\mathbb {E}}}\bigg [ \int _0^{\theta _r} e^{-\delta s}f({X}(s-))\cdot dY(s) - \int _0^{\theta _r} e^{-\delta s}g({X}(s-))\cdot dZ(s) + e^{-\delta \theta _r} \phi ({X}(\theta _r))\bigg ].\qquad \quad \end{aligned}$$

(A.9)

By the Dynkin formula, we obtain

$$\begin{aligned} \phi (x^0)= & {} {{\mathbb {E}}}e^{-\delta \theta _r} \phi ({X}(\theta _r)) - {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds\nonumber \\&+\, {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} \nabla \phi ({X}(s)) \cdot dY^c (s) - {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} \nabla \phi ({X}(s)) \cdot dZ^c (s) \nonumber \\&-\, {{\mathbb {E}}}\sum \limits _{0\le s\le \theta _r} e^{-\delta s} \Big [ \phi ({X}(s) )- \phi ({X}(s-)) \Big ]. \end{aligned}$$

(A.10)

A combination of (A.9) and (A.10) leads to

$$\begin{aligned} \begin{array}{ll} 0 &{}\!\displaystyle \ge {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s}f({X}(s-))\cdot dY(s) - {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s}g({X}(s-))\cdot dZ(s)\\ &{}\!\displaystyle \qquad + {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds\\ &{}\!\displaystyle \qquad - {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} \nabla ({X}(s)) \cdot dY^c (s) + {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} \nabla ({X}(s)) \cdot dZ^c (s) \\ &{}\!\displaystyle \qquad + {{\mathbb {E}}}\sum \limits _{0\le s\le \theta _r} e^{-\delta s} \Big [ \phi ({X}(s))- \phi ({X}(s-)) \Big ]. \end{array} \end{aligned}$$

(A.11)

First, we take $\eta =0$; that is, $Y(t)=Z(t)=0$ for any $t\ge 0$. Then $\theta >0$ almost surely (a.s.) and (A.11) can be rewritten as

$$\begin{aligned} \begin{array}{ll} 0&\!\displaystyle \ge {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds. \end{array} \end{aligned}$$

(A.12)

We suppose that $({\mathcal {L}}-\delta )\phi (x^0)>0$. Then we can choose a sufficiently small $\varepsilon >0$ such that $({\mathcal {L}}-\delta )\phi (x)>0$ for any $x\in B_\varepsilon (x^0)$. Note that we can find such an $\varepsilon$ while still making sure that the condition from the beginning of the proof holds - one just needs to take the minimum of the two small numbers. As a result, $({\mathcal {L}}-\delta )\phi (X(s))>0$ for any $t\in [0, \theta )$. It follows that $\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds>0$ and therefore,

$$\begin{aligned} {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds>0, \end{aligned}$$

which contradicts (A.12). This proves that

$$\begin{aligned} ({\mathcal {L}}-\delta )\phi (x^0)\le 0. \end{aligned}$$

(A.13)

Next, we fix $i\in \{1, \dots , d\}$ and take $\eta \in (0, \varepsilon )$, $Y_i(t)=Y_i(0)=\eta$ for all $t\ge 0$, and $Y_j(t)=Z_i(t)=Z_j(t)=0$ for all $j\ne i$ and $t\ge 0$. Then (A.11) reduces to

$$\begin{aligned} {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds + f_i(x^0)\eta + \phi (x^0- \eta \mathbf{e}_i)-\phi (x^0)\le 0. \end{aligned}$$

Now sending $r\rightarrow 0$, we have

$$\begin{aligned} f_i(x^0)\eta + \phi (x^0- \eta \mathbf{e}_i)-\phi (x^0)\le 0. \end{aligned}$$

By dividing the above inequality by $\eta$ and letting $\eta \rightarrow 0$, we obtain

$$\begin{aligned} f_i(x^0)-\partial \phi /\partial x_i (x^0)\le 0. \end{aligned}$$

(A.14)

Now we take $\eta \in (0, \varepsilon )$, $Z_i(t)=\eta$ for all $t\ge 0$, and $Z_j(t)=Y_i(t)=Y_j(t)=0$ for all $j\ne i$ and $t\ge 0$. Then (A.11) reduces to

$$\begin{aligned} {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} ({\mathcal {L}}-\delta ) \phi ({X}(s))ds - g_i(x^0)\eta + \phi (x^0+ \eta \mathbf{e}_i)-\phi (x^0)\le 0. \end{aligned}$$

Now sending $r\rightarrow 0$, we have

$$\begin{aligned} -g_i(x^0)\eta + \phi (x^0- \eta \mathbf{e}_i)-\phi (x^0)\le 0. \end{aligned}$$

Finally, dividing the above inequality by $\eta$ and letting $\eta \rightarrow 0$, we arrive at

$$\begin{aligned} \partial \varphi /\partial x_i(x^0)-g_i(x^0)\le 0. \end{aligned}$$

(A.15)

Now (A.8) follows by combining (A.13), (A.14), and (A.15). $\square$

Lemma A.3

Suppose (Y, Z) is an admissible strategy and let X be the resulting population process. Define the random variable $\lambda$ as follows. If $X(\theta )=X(\theta -)$ then $\lambda =0$, while if $X(\theta )\notin \overline{B_\varepsilon (x^0)}$, then let $\lambda$ be a positive number in (0, 1] such that

$$\begin{aligned} {X}(\theta -) +\lambda ({X}(\theta ) - {X}(\theta -) )\in \partial {B_\varepsilon (x^0)}. \end{aligned}$$

Then there is a positive number $\kappa _0>0$ such that

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle {{\mathbb {E}}}\Bigg [ \int _0^{\theta } e^{-\delta s} ds + \lambda e^{-\delta \theta } {\hbox {1{}1}}\cdot \Delta Z(\theta ) + \lambda e^{-\delta \theta } {\hbox {1{}1}}\cdot \Delta Y(\theta ) \\ &{}\!\displaystyle \qquad + \int _0^{\theta -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s)\Bigg ]\ge \kappa _0, \end{array} \end{aligned}$$

(A.16)

where ${\hbox {1{}1}}(x)=(1,\dots ,1)'$ for all $x\in {\bar{S}}$.

Proof

Recall that $\theta _r=\theta \wedge r$ for any positive number r. Define

$$\begin{aligned} {X}_\lambda ^r:={X}(\theta _r-) +\lambda ({X}(\theta _r) - {X}(\theta _r-) )={X}(\theta _r-)-\lambda (\Delta Y(\theta _r )- \Delta Z(\theta _r)). \end{aligned}$$

It can be seen that $X_\lambda ^r\in \overline{B_\varepsilon (x^0)}$ for any $r>0$. We consider the function ${\widetilde{\Phi }} (x)= |x-x^0|^2-\varepsilon ^2$ for $x \in B_\varepsilon (x^0)$. It follows that

$$\begin{aligned} ({\mathcal {L}}-\delta ){\widetilde{\Phi }}(x)=2(x-x^0)\cdot b(x) +\sum \limits _{i=1}^d \sigma _{ii}^2(x)-\delta (|x-x^0|^2-\varepsilon ^2). \end{aligned}$$

Since ${\widetilde{\Phi }}(\cdot ), b(\cdot )$, and $\sigma (\cdot )$ are continuous, it is obvious that

$$\begin{aligned} |({\mathcal {L}}-\delta ){\widetilde{\Phi }}(x)|\le K<\infty \end{aligned}$$

for some positive constant K. Note that K depends only on $x^0, \varepsilon , \delta$ and bounds on $b(\cdot ), \sigma (\cdot )$. Let $K_0=\dfrac{1}{K+2\varepsilon }$ and define $\Phi (x)=K_0 {\widetilde{\Phi }}(x)$ for $x\in B_\varepsilon (x^0)$. Then it follows immediately that

$$\begin{aligned} |({\mathcal {L}}-\delta )\Phi (x)|<1 \quad \text {for } x\in B_\varepsilon (x^0). \end{aligned}$$

(A.17)

Moreover, we have

$$\begin{aligned} \nabla \Phi (x) \cdot {\hbox {1{}1}}=2K_0 \cdot (x-x_0)\ge -1. \end{aligned}$$

(A.18)

By virtue of the Dynkin formula, we have

$$\begin{aligned}&{{\mathbb {E}}}e^{-\delta \theta _r} \Phi ({X}(\theta _r-)-\Phi (x^0) \nonumber \\&\quad = {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} ({\mathcal {L}}-\delta ) \Phi ({X}(s))ds - {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} \nabla \Phi ({X}(s)) \cdot dY^c (s)\nonumber \\&\qquad + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} \nabla \Phi ({X}(s)) \cdot dZ^c (s) + {{\mathbb {E}}}\sum \limits _{0\le s< \theta _r} e^{-\delta s} \Big [ \Phi ({X}(s) )- \Phi ({X}(s-)) \Big ] .\nonumber \\ \end{aligned}$$

(A.19)

By virtue of (A.18), we have

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle \Phi ({X}(s) )- \Phi ({X}(s-))\\ &{}\displaystyle \ = ({X}(s) - ({X}(s-) )\cdot \nabla \Phi (P(s)) \\ &{}\displaystyle \ = (-\Delta Y (s) + \Delta Z(s) ) \cdot \nabla \Phi (P(s)),\\ &{}\displaystyle \ \le {\hbox {1{}1}}\cdot (\Delta Y (s) + \Delta Z(s) ) , \end{array} \end{aligned}$$

(A.20)

where P(s) is a point on the line connecting X(s) and $X(s-)$. Hence it follows from (A.17)–(A.20) that

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle {{\mathbb {E}}}e^{-\delta \theta _r} \Phi ({X}(\theta _r-)))-\Phi (x^0)) \\ &{}\!\displaystyle \quad \le {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s}ds + E \int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dY^c (s)+{{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dZ^c (s)\\ &{}\!\displaystyle \qquad + {{\mathbb {E}}}\sum \limits _{0\le s< \theta _r} e^{-\delta s} {\hbox {1{}1}}\cdot \big [ \Delta Y (s) + \Delta Z(s) \big ]\\ &{}\!\displaystyle \quad = {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s}ds + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dY (s) + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dZ(s) . \end{array} \end{aligned}$$

(A.21)

We also have

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle \Phi ({X}(\theta _r-) )- \Phi ({X}_\lambda ^r)\\ &{}\displaystyle \ = ({X}(\theta _r-) - {X}_\lambda ^r )\cdot \nabla \Phi (P_0)\\ &{}\displaystyle \ = \lambda (\Delta Y (\theta _r) - \Delta Z(\theta _r) ) \cdot \nabla \Phi (P_0)\\ &{}\displaystyle \ \ge -\lambda {\hbox {1{}1}}\cdot (\Delta Y (\theta _r) + \Delta Z(\theta _r) ), \end{array} \end{aligned}$$

(A.22)

where $P_0$ is a point on the line segment connecting ${X}(\theta _r-)$ and ${X}_\lambda ^r$. Thus,

$$\begin{aligned} {{\mathbb {E}}}e^{-\delta \theta _r} \Phi ({X}_\lambda ^r ) - {{\mathbb {E}}}e^{-\delta \theta _r} \Phi ({X}(\theta _r-) )\le \lambda {{\mathbb {E}}}e^{-\delta \theta _r} {\hbox {1{}1}}\cdot (\Delta Y (\theta _r) + \Delta Z(\theta _r) ).\qquad \end{aligned}$$

(A.23)

Combining (A.21) and (A.23), we have

$$\begin{aligned}&{{\mathbb {E}}}e^{-\delta \theta _r} \Phi ({X}_\lambda ^r )-\Phi (x^0)\nonumber \\&\quad \ \le {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s}ds + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dY (s) + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} {\hbox {1{}1}}\cdot dZ(s) \nonumber \\&\qquad + \lambda {{\mathbb {E}}}e^{-\delta \theta _r} {\hbox {1{}1}}\cdot (\Delta Y (\theta _r) + \Delta Z(\theta _r) ). \end{aligned}$$

(A.24)

By letting $r\rightarrow \infty$, we arrive at

$$\begin{aligned}&\!\displaystyle {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s}ds + {{\mathbb {E}}}\int _0^{\theta -} e^{-\delta s} {\hbox {1{}1}}\cdot dY (s) + {{\mathbb {E}}}\int _0^{\theta -} e^{-\delta s} {\hbox {1{}1}}\cdot dZ(s) \nonumber \\&\!\displaystyle \qquad + \lambda {{\mathbb {E}}}e^{-\delta \theta } {\hbox {1{}1}}\cdot (\Delta Y (\theta ) + \Delta Z(\theta ) )\ge {{\mathbb {E}}}e^{-\delta \theta } \Phi ({X}_\lambda )-\Phi (x^0). \end{aligned}$$

(A.25)

If ${{\mathbb {P}}}(\theta =\infty )>0$, then ${{\mathbb {E}}}e^{-\delta \theta }\Phi ({X}_\lambda )=0$. Otherwise, $\theta <\infty$ a.s. and in that case, since $\Phi ({X}_\lambda )\in \partial B_\varepsilon (x^0)$, $\Phi ({X}_\lambda )=0.$ Also, it is clear that $\Phi (x^0)= -K_0 \varepsilon ^2.$ Thus, we obtain

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s}ds + {{\mathbb {E}}}\int _0^{\theta -} e^{-\delta s} {\hbox {1{}1}}\cdot dY (s) + {{\mathbb {E}}}\int _0^{\theta -} e^{-\delta s} {\hbox {1{}1}}\cdot dZ(s)\\ &{}\!\displaystyle \qquad + \lambda {{\mathbb {E}}}e^{-\delta \theta } {\hbox {1{}1}}\cdot (\Delta Y (\theta ) + \Delta Z(\theta ) ) \ge K_0\varepsilon ^2>0. \end{array} \end{aligned}$$

Set $\kappa _0:=K_0\varepsilon ^2$ and note that the above establishes (A.16). $\square$

Proposition A.4

Let Assumption 2.3 be satisfied and assume that $V(x)<\infty$ for $x\in {{\bar{S}}}$. The value function V is a viscosity supersolution of the system of quasi-variational inequalities (A.7); that is, for any $x^0\in S$ and any function $\varphi \in C^2(S)$ satisfying

$$\begin{aligned} (V-\varphi )(x)\le (V-\varphi )(x^0)=0, \end{aligned}$$

(A.26)

for all x in a neighborhood of $x^0$, we have

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\varphi (x^0), f_i(x^0)-\dfrac{\partial \varphi }{\partial x_i}(x^0), \dfrac{\partial \varphi }{\partial x_i}(x^0)-g_i(x^0)\bigg \}\ge 0. \end{aligned}$$

(A.27)

Proof

Let $x^0\in S$ and suppose $\varphi (\cdot )\in C^2(S)$ satisfies (A.26) for all x in a neighborhood of $x^0$.

We argue by contradiction. Suppose that (A.27) does not hold. Then there exists a constant $A>0$ such that

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\varphi (x^0), f_i(x^0)-\dfrac{\partial \varphi }{\partial x_i}(x^0), \dfrac{\partial \varphi }{\partial x_i}(x^0)-g_i(x^0)\bigg \}\le -2A< 0.\qquad \end{aligned}$$

(A.28)

Let $\varepsilon >0$ be small enough so that $\overline{B_\varepsilon (x^0)}\subset S$ and for any $x\in \overline{B_\varepsilon (x^0 )}$, $\varphi (x)\ge V(x)$ and

$$\begin{aligned} \max \limits _{i} \bigg \{({\mathcal {L}}-\delta )\varphi (x), f_i(x)-\dfrac{\partial \varphi }{\partial x_i}(x), \dfrac{\partial \varphi }{\partial x_i}(x)-g_i(x)\bigg \}\le -A< 0. \end{aligned}$$

(A.29)

Let $(Y, Z)\in {\mathcal {A}}_{x^0}$ and X be the corresponding harvested process. Recall that $\theta =\inf \{t\ge 0: {X}(t)\notin B_\varepsilon (x^0) \}$ and $\theta _r=\theta \wedge r$ for any $r>0$. It follows from the Dynkin formula that

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle {{\mathbb {E}}}e^{-\delta \theta _r} \varphi ({X}(\theta _r-)-\varphi (x^0)) \\ &{}\!\displaystyle \quad = {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} ({\mathcal {L}}-\delta ) \varphi ({X}(s))ds - {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} \nabla \varphi ({X}(s)) \cdot dY^c (s)\\ &{}\!\displaystyle \quad + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} \nabla \varphi ({X}(s)) \cdot dZ^c (s) + {{\mathbb {E}}}\sum \limits _{0\le s< \theta _r} e^{-\delta s} \Big [ \varphi ({X}(s) )- \varphi ({X}(s-)) \Big ] . \end{array}\nonumber \\ \end{aligned}$$

(A.30)

By virtue of Lemma A.1, for any $s\in [0, \theta _r)$, there exist ${\widehat{X}}(s)\in \mathbb {R}^d$ and ${\widetilde{X}}(s)\in \mathbb {R}^d$ such that ${\widehat{X}}(s)\le X(s)$, ${\widetilde{X}}(s)\le X(s)$, and

$$\begin{aligned} \varphi ({X}(s) )- \varphi ({X}(s-))\le -\Delta Y(s)\cdot \nabla \varphi ({\widehat{X}}(s)) + \Delta Z(s)\cdot \nabla \varphi ({\widetilde{X}}(s)). \end{aligned}$$

This, together with the monotonicity of the functions f, g, and equation (A.29) imply that

$$\begin{aligned} \begin{array}{ll} \varphi ({X}(s) )- \varphi ({X}(s-))&{}\!\displaystyle \le ( -f({X}(s)) -A{\hbox {1{}1}})\cdot \Delta Y(s)\\ &{}\!\displaystyle \qquad + ( g({X}(s)) -A {\hbox {1{}1}})\cdot \Delta Z(s). \end{array} \end{aligned}$$

Hence it follows from (A.29) and (A.30) that

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle {{\mathbb {E}}}e^{-\delta \theta _r} \varphi ({X}(\theta _r-))-\varphi (x^0)) \\ &{}\!\displaystyle \quad \le {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} (-A)ds + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} (-f ({X}(s))-A{\hbox {1{}1}}) \cdot dY^c (s)\\ &{}\!\displaystyle \qquad + {{\mathbb {E}}}\int _0^{\theta _r-} e^{-\delta s} (g({X}(s)) -A{\hbox {1{}1}}) \cdot dZ^c (s) \\ &{}\qquad + {{\mathbb {E}}}\sum \limits _{0\le s< \theta _r} e^{-\delta s} ( -f({X}(s)) -A{\hbox {1{}1}})\cdot \Delta Y(s)\\ &{}\!\displaystyle \qquad + {{\mathbb {E}}}\sum \limits _{0\le s< \theta _r} e^{-\delta s} ( g({X}(s)) -A{\hbox {1{}1}})\cdot \Delta Z(s)\\ &{}\!\displaystyle \quad =- {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} f ({X}(s))\cdot dY(s) \\ &{}\qquad + {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} g ({X}(s))\cdot dZ(s)-A {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} ds \\ &{}\!\displaystyle \qquad - A {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) - A {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} {\hbox {1{}1}}\cdot dZ(s). \end{array} \end{aligned}$$

(A.31)

Therefore,

$$\begin{aligned} \begin{array}{ll} \varphi (x^0) &{}\!\displaystyle \ge {{\mathbb {E}}}e^{-\delta \theta _r} \varphi ({X}(\theta _r-)) + {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} f ({X}(s-))\cdot dY(s) \\ &{}\!\displaystyle \quad - {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} g ({X}(s-))\cdot dZ(s)\\ &{}\!\displaystyle \quad + A {{\mathbb {E}}}\bigg [ \int _0^{\theta _r} e^{-\delta s} ds + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s) \bigg ]. \end{array}\nonumber \\ \end{aligned}$$

(A.32)

We are in a position to apply Lemma A.3. To this end, recall from Lemma A.3 that $\lambda$ is a random variable such that if $X(\theta )=X(\theta -)$, then $\lambda =0$; and if $X(\theta -)\ne X(\theta )$, then $\lambda$ is the positive number in (0, 1] such that

$$\begin{aligned} {X}_\lambda ={X}(\theta -) +\lambda ({X}(\theta ) - {X}(\theta -) )={X}(\theta -)-\lambda (\Delta Y(\theta )- \Delta Z(\theta ))\in \partial {B_\varepsilon (x^0)}. \end{aligned}$$

Note that $\lambda$ is independent of r. Also recall that

$$\begin{aligned} {X}_\lambda ^r={X}(\theta _r-) +\lambda ({X}(\theta _r) - {X}(\theta _r-) )={X}(\theta _r-)-\lambda (\Delta Y(\theta _r )- \Delta Z(\theta _r ))\in \overline{B_\varepsilon (x^0)}. \end{aligned}$$

Using the same argument as the one in Lemma A.1, we obtain

$$\begin{aligned} \begin{array}{ll} \varphi ( {X}(\theta _r-) ) - \varphi ( {X}_\lambda ^r)&{}\!\displaystyle \ge \lambda [f({X}(\theta _r-)) +A {\hbox {1{}1}}]\cdot \Delta Y(\theta _r) \\ &{}\!\displaystyle \quad + \lambda [A{\hbox {1{}1}}- g({X}(\theta _r-))]\cdot \Delta Z(\theta _r). \end{array} \end{aligned}$$

(A.33)

Combining (A.32) and (A.33), we have

$$\begin{aligned} \begin{array}{ll} &{}\!\displaystyle V(x^0) = \varphi (x^0) \\ &{}\!\displaystyle \quad \ge {{\mathbb {E}}}e^{-\delta s} \varphi ( {X}_\lambda ^r) + {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} f ({X}(s-))\cdot dY(s)\\ &{}\!\displaystyle \qquad - {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} g ({X}(s-))\cdot dZ(s)\\ &{}\!\displaystyle \qquad + A {{\mathbb {E}}}\bigg [ \int _0^{\theta _r} e^{-\delta s} ds + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s) \bigg ] \\ &{}\!\displaystyle \qquad + \lambda {{\mathbb {E}}}e^{-\delta \theta _r} [f({X}(\theta _r-)) +A{\hbox {1{}1}}]\cdot \Delta Y(\theta _r) + \lambda {{\mathbb {E}}}e^{-\delta \theta _r} [A{\hbox {1{}1}}- g({X}(\theta _r-))]\cdot \Delta Z(\theta _r). \end{array}\nonumber \\ \end{aligned}$$

(A.34)

Since ${X}_\lambda ^r\in \overline{B_\varepsilon (x^0)}$, $\varphi ({X}_\lambda ^r)\ge V({X}_\lambda ^r)$. On the other hand, it follows from (2.6) that

$$\begin{aligned} \begin{array}{ll} V({X}_\lambda ^r)&{}\!\displaystyle \ge V({X}(\theta _r))\\ &{}\!\displaystyle \quad + (1-\lambda )\Delta Y(\theta _r)\cdot f({X}_\lambda ^r) -(1-\lambda ) \Delta Z(\theta _r)\cdot g( {X}_\lambda ^r)\\ &{}\!\displaystyle \ge \quad V({X}(\theta _r))\\ &{}\!\displaystyle \quad + (1-\lambda )\Delta Y(\theta _r)\cdot f({X}(\theta _r-)) -(1-\lambda ) \Delta Z(\theta _r)\cdot g( {X}(\theta _r-)). \end{array} \end{aligned}$$

(A.35)

By (A.34) and (A.35) we note that

$$\begin{aligned} V(x^0)&\ge {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} f ({X}(s-))\cdot dY(s) - {{\mathbb {E}}}\int _0^{\theta _r -} e^{-\delta s} g ({X}(s-))\cdot dZ(s)\nonumber \\&\quad + A {{\mathbb {E}}}\bigg [ \int _0^{\theta _r} e^{-\delta s} ds + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s) \bigg ] + {{\mathbb {E}}}e^{-\delta \theta _r} V({X}(\theta _r))\nonumber \\&\quad + \lambda {{\mathbb {E}}}e^{-\delta \theta _r} [f({X}(\theta _r-)) +A {\hbox {1{}1}}]\cdot \Delta Y(\theta _r) + \lambda \ e^{-\delta \theta _r} [A{\hbox {1{}1}}- g({X}(\theta _r-))]\cdot \Delta Z(\theta _r)\nonumber \\&\quad + (1-\lambda ){{\mathbb {E}}}e^{-\delta \theta _r} \Delta Y(\theta _r)\cdot f({X}(\theta _r-)) -(1-\lambda ) {{\mathbb {E}}}e^{-\delta \theta _r} \Delta Z(\theta _r)\cdot g( {X}(\theta _r))\nonumber \\&\ge {{\mathbb {E}}}e^{-\delta \theta _r} V({X}(\theta _r))+{{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} f ({X}(s-)\cdot dY(s) - {{\mathbb {E}}}\int _0^{\theta _r} e^{-\delta s} g ({X}(s-))\cdot dZ(s)\nonumber \\&\quad + A {{\mathbb {E}}}\Big [ \int _0^{\theta _r} e^{-\delta s} ds + \lambda e^{-\delta \theta _r} {\hbox {1{}1}}\cdot \Delta Z(\theta _r) + \lambda e^{-\delta \theta _r} {\hbox {1{}1}}\cdot \Delta Y(\theta _r)\nonumber \\&\quad + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta _r -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s) \Big ]. \end{aligned}$$

(A.36)

Letting $r\rightarrow \infty$, we have

$$\begin{aligned} V(x^0)&\ge {{\mathbb {E}}}e^{-\delta \theta } V({X}(\theta ))+ {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s} f ({X}(s-)\cdot dY(s)\nonumber \\&\quad - {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s} g ({X}(s-))\cdot dZ(s)\nonumber \\&\quad + A {{\mathbb {E}}}\Big [ \int _0^{\theta } e^{-\delta s} ds + \lambda e^{-\delta \theta } {\hbox {1{}1}}\cdot \Delta Z(\theta ) + \lambda e^{-\delta \theta } {\hbox {1{}1}}\cdot \Delta Y(\theta ) \nonumber \\&\quad + \int _0^{\theta -} e^{-\delta s}{\hbox {1{}1}}\cdot dY(s) + \int _0^{\theta -} e^{-\delta s}{\hbox {1{}1}}\cdot dZ(s)\Big ]. \end{aligned}$$

(A.37)

Using (A.16) and (A.37), we arrive at

$$\begin{aligned} V(x^0)\ge & {} {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s} f ({X}(s-))\cdot dY(s) - {{\mathbb {E}}}\int _0^{\theta } e^{-\delta s} g ({X}(s-))\cdot dZ(s)\nonumber \\&+\, {{\mathbb {E}}}e^{-\delta \theta } V({X}(\theta )) +A\kappa _0. \end{aligned}$$

(A.38)

Taking the supremum over $(Y, Z)\in {\mathcal {A}}_{x^0}$, it follows that

$$\begin{aligned} \begin{array}{ll} V(x^0)&{}\!\displaystyle \, \ge \sup \limits _{{\mathcal {A}}_{x^0}} {{\mathbb {E}}}\bigg [ \int _0^{\theta } e^{-\delta s} f ({X}(s-))\cdot dY(s) -\int _0^{\theta } e^{-\delta s} g ({X}(s-))\cdot dZ(s) \\ &{}\!\displaystyle \quad + {{\mathbb {E}}}e^{-\delta \theta } V({X}(\theta )) +A\kappa _0\bigg ]. \end{array}\qquad \end{aligned}$$

(A.39)

In view of the dynamic programming principle, (A.39) can be rewritten as

$$\begin{aligned} V(x^0)\ge V(x^0) +A\kappa _0 > V(x^0), \end{aligned}$$

which is a contradiction. As a result (A.27) has to hold, i.e. V is viscosity supersolution of (A.7). $\square$

Summarizing what have obtained thus far, we state the following result.

Theorem A.5

Let Assumption 2.3 be satisfied and suppose $V(x)<\infty$ for $x\in {S}$. The value function V is a viscosity subsolution and also a viscosity supersolution, and hence a viscosity solution, of the system of quasi-variational inequalities (A.7).

Appendix B: Numerical algorithm

1.1 B.1: Transition probabilities and local consistency

We use the notation defined in Sect. 2.1. To proceed, we state one more assumption below, which will be used to ensure the validity of transition probabilities $p^h(x, y|\pi )$. However, it is not an essential assumption. There are several alternatives to handle the cases when Assumption B.1 fails. We refer the reader to page 1013 in the book by Kushner (1990) for a detailed discussion. Define for any $x\in {\bar{S}}$ the covariance matrix $a(x)= \sigma (x)\sigma '(x)$.

Assumption B.1

For any $i=1, \dots , d$ and $x\in {{\bar{S}}}$,

$$\begin{aligned} a_{ii}(x)-\sum \limits _{j: j\ne i}\big |a_{ij}(x)\big |\ge 0. \end{aligned}$$

Let ${{\mathbb {E}}}^{h, \pi }_{x, n}$, ${\mathbb Cov}^{h, \pi }_{x, n}$ denote the conditional expectation and covariance given by

$$\begin{aligned} \{X_m^h, \pi _m^h, m\le n, X_n^h=x, \pi ^h_n=\pi \}, \end{aligned}$$

respectively. Our objective in this subsection is to define transition probabilities $p^h (x, y | \pi )$ so that the controlled Markov chain $\{X^h_n\}$ is locally consistent with respect to the diffusion (2.2) in the sense that the following conditions hold:

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle {{\mathbb {E}}}^{h, 0}_{x, n}\Delta X_n^h = {b}(x) \Delta t^h(x) + o(\Delta t^h(x)),\\ &{}\displaystyle Cov^{h, 0}_{x, n}\Delta X_n^h = a(x)\Delta t^h(x) + o(\Delta t^h(x)),\\ &{}\displaystyle \sup \limits _{n, \ \omega } |\Delta X_n^h| \rightarrow 0 \ \hbox { as } \ h \rightarrow 0. \end{array} \end{aligned}$$

(B.1)

To this end, using the procedure introduced by Kushner (1990), we define the approximation to the first and the second derivatives of V by a finite difference method using the step size $h>0$ for the state variable. Afterwards, we plug in all the approximations into the first part of system (A.7), combine similar terms and divide by the coefficient of $V^h(x)$. The transition probabilities are the coefficients of the resulting equation. For $x\in S_h$, define

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle Q_h (x)=\sum \limits _{i=1}^d a_{ii}(x) -\sum \limits _{i, j: i\ne j}\dfrac{1}{2}|a_{ij}(x)| +h\sum \limits _{i=1}^d |b_i(x)|,\\ &{}\displaystyle \Delta t^h(x)=\dfrac{h^2}{Q_h(x)},\\ &{}\displaystyle p^h (x, x+h\mathbf{e_i}|\pi =0) = \dfrac{a_{ii}(x)/2-\sum \limits _{j: j\ne i}|a_{ij}(x)|/2+b_{i}^+(x) h }{Q_h (x)}, \\ &{}\displaystyle p^h (x, x-h \mathbf{e_i}) | \pi =0) = \dfrac{a_{ii}(x)/2-\sum \limits _{j: j\ne i}|a_{ij}(x)|/2+b_{i}^-(x) h}{Q_h (x)}, \\ &{}\displaystyle p^h ( x, x+h\mathbf{e_i}+h \mathbf{e_j}| \pi =0) = p^h ( x, x-h\mathbf{e_i}-h\mathbf{e_j}| \pi =0) = \dfrac{a_{{ij}}^+(x)}{2Q_h (x)}, \\ &{}\displaystyle p^h ( x, x+h\mathbf{e_i}-h \mathbf{e_j}| \pi =0) = p^h ( x, x-h\mathbf{e_i}+ h \mathbf{e_j}| \pi =0 )= \dfrac{a_{{ij}}^-(x)}{2Q_h (x)}, \end{array} \end{aligned}$$

(B.2)

where for a real number c, $c^+=\max \{c, 0\}$, $c^-=-\min \{0, c\}$; that is, $c=c^+$ if $c\ge 0$ and $c=-c^-$ if $c<0$. Set $p^h (x, y|\pi =0)=0$ for all unlisted values of $y\in S^h$. Assumption B.1 guarantees that the transition probabilities in (B.2) are well-defined. At the seeding and harvesting steps, we define

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle p^h( x, x - h \mathbf{e_i}| \pi = i)=1,\\ &{}\displaystyle p^h( x, x + h \mathbf{e_i}| \pi =- i)=1. \end{array} \end{aligned}$$

(B.3)

Thus, $p^h (x, y|\pi =\pm i)=0$ for all nonlisted values of $y\in S^h$. Using the above transition probabilities, we can check whether the locally consistent conditions of $\{X^h_n\}$ in (B.1) are satisfied.

1.2 B.2: Continuous-time interpolation and time rescaling

The convergence result is based on a continuous-time interpolation of the chain, which will be constructed to be piecewise constant on the time interval $[t^h_n, t^h_{n+1}), n\ge 0$. For use in this construction, we define $n^h(t)=\max \{n: t^h_n\le t\}, t\ge 0$. We first define discrete time processes associated with the controlled Markov chain as follows. Let $Y^h_0=Z^h_0=B^h_0=M^h_0=0$ and define for $n\ge 1$,

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle Y^h_n = \sum \limits _{m=0}^{n-1}\Delta Y^h_m, \quad Z^h_n = \sum \limits _{m=0}^{n-1}\Delta Z^h_m,\\ &{}\displaystyle B^h_n = \sum \limits _{m=0}^{n-1} I_{\{\pi _m^h=0\}} {{\mathbb {E}}}^{h}_m \Delta \xi ^h_m, \quad M^h_n = \sum \limits _{m=0}^{n-1} (\Delta \xi ^h_m - {{\mathbb {E}}}^{h}_m \Delta X_m)I_{\{\pi _m^h=0\}}. \end{array} \end{aligned}$$

(B.4)

The piecewise constant interpolations, denoted by $(X^h(\cdot ), Y^h(\cdot ), Z^h(\cdot ), B^h(\cdot ), M^h(\cdot ))$ are naturally defined as

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle X^h(t) = X^h_{n^h(t)},\\ &{}\displaystyle Y^h(t) = Y^h_{n^h(t)}, \quad Z^h(t) = Z^h_{n^h(t)},\\ &{}\displaystyle B^h(t) = B^h_{n^h(t)}, \quad M^h(t) = M^h_{n^h(t)}, \quad t\ge 0. \end{array} \end{aligned}$$

(B.5)

Define ${\mathcal {F}}^h(t)=\sigma \{X^h(s), Y^h(s), Z^h(s): s\le t\}={\mathcal {F}}^h_{n^h(t)}$. Using the representation of diffusion, harvesting, and seeding steps in (2.17), we obtain

$$\begin{aligned} \begin{array}{ll} X^h_n = x + \sum \limits _{m=0}^{n-1} \Delta X_m^h {\hbox {1{}1}}_{\{ \pi ^h_m\le -1\}} + \sum \limits _{m=0}^{n-1} \Delta X_m^h {\hbox {1{}1}}_{\{ \pi ^h_m\ge 1\}}+ \sum \limits _{m=0}^{n-1} \Delta X_m^h {\hbox {1{}1}}_{\{ \pi ^h_m=0\}} \end{array} \end{aligned}$$

(B.6)

This implies

$$\begin{aligned} X^h(t) = x + B^h(t) + M^h(t) -Y^h(t)+Z^h(t). \end{aligned}$$

(B.7)

Recall that $\Delta t^h_m = h^2/Q_h(X^h_m)$ if $\pi ^h_m=0$ and $\Delta t^h_m = 0$ if $\pi ^h_m\ge 1$ or $\pi ^h_m\le -1$. It follows that

$$\begin{aligned} \begin{array}{ll} B^h(t) &{}\!\displaystyle = \sum \limits _{m=0}^{n^h(t)-1} b (X^h_m)\Delta t^h_m\\ &{}\!\displaystyle =\int _0^t b (X^h(s)) ds-\int _{t^h_{n^h(t)}}^t b (X^h(s)) ds\\ &{}\!\displaystyle = \int _0^t b (X^h(s)) ds + \varepsilon ^h_1(t), \end{array} \end{aligned}$$

(B.8)

with $\{\varepsilon _1^h(\cdot )\}$ being an ${\mathcal {F}}^h(t)$-adapted process satisfying

$$\begin{aligned} \lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{{\mathbb {E}}}|\varepsilon _1^h(t)|=0 \quad \text {for any }0<T_0<\infty . \end{aligned}$$

We now attempt to represent $M^h(\cdot )$ in a form similar to the diffusion term in (2.2). Factor

$$\begin{aligned} a(x)= \sigma (x)\sigma '(x)=P(x)D^2(x)P'(x), \end{aligned}$$

where $P(\cdot )$ is an orthogonal matrix, $D(\cdot )=\mathrm{diag}\{r_1(\cdot ), ..., r_d (\cdot )\}$. Without loss of generality, we suppose that $\inf \limits _{x}r_i(x)>0$ for all $i=1, ..., d$. Define $D_0(\cdot )=\mathrm{diag}\{1/r_1(\cdot ), ..., 1/r_d (\cdot )\}$.

Remark B.2

In the argument above, for simplicity, we assume that the diffusion matrix (a(x)) is nondegenerate. If this is not the case, we can use the trick from (Kushner and Dupuis 1992, pp. 288–289) to establish equation (B.10).

Define $W^h(\cdot )$ by

$$\begin{aligned} \begin{array}{ll} W^h(t) &{}\!\displaystyle = \int _0^t D_0 (X^h(s)) P' (X^h(s))dM^h(s)\\ &{}\!\displaystyle = \sum \limits _{m=0}^{n^h(t)-1} D_0 (X^h_m) P' (X^h_m)(\Delta \xi ^h_m -{{\mathbb {E}}}^{h}_m \Delta \xi ^h_m)I_{\{ \pi ^h_m=0\}}. \end{array} \end{aligned}$$

(B.9)

Then we can write

$$\begin{aligned} M^h(t) =\int _0^t \sigma (\xi ^h(s)) dW^h(s) + \varepsilon _2^h(t), \end{aligned}$$

(B.10)

with $\{\varepsilon _2^h(\cdot )\}$ being an ${\mathcal {F}}^h(t)$-adapted process satisfying

$$\begin{aligned} \lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{{\mathbb {E}}}|\varepsilon _2^h(t)|=0 \quad \text {for any }0<T_0<\infty . \end{aligned}$$

Using (B.8) and (B.10), we can write (B.7) as

$$\begin{aligned} X^h(t) = x + \int _0^t b (X^h(s)) ds + \int _0^t \sigma (X^h(s)) dW^h(s) -Y^h(t)+Z^h(t)+\varepsilon ^h(t),\nonumber \\ \end{aligned}$$

(B.11)

where $\varepsilon ^h(\cdot )$ is an ${\mathcal {F}}^h(t)$-adapted process satisfying

$$\begin{aligned} \lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{{\mathbb {E}}}|\varepsilon ^h(t)|=0 \quad \text {for any }0<T_0<\infty . \end{aligned}$$

The objective function from (2.20) can be rewritten as

$$\begin{aligned} J^h(x, Y^h, Z^h)= {{\mathbb {E}}}\int _0^{\infty } e^{-\delta s}\Big [ f(X^h(s-)) \cdot dY^{h}(s)-g(X^h(s-))\cdot dZ^{h}(s)\Big ].\nonumber \\ \end{aligned}$$

(B.12)

Time rescaling. Next we will introduce a “stretched-out” time scale. This is similar to the approach previously used by Kushner and Martins (1991) and Budhiraja and Ross (2007) for singular control problems. Using the new time scale, we can overcome the possible non-tightness of the family of processes $\{Y^h(\cdot ), Z^h(\cdot )\}_{h>0}$.

Define the rescaled time increments $\{\Delta \widehat{t}_n^h: n=0, 1, ...\}$ by

$$\begin{aligned} \begin{array}{ll} &{}\displaystyle \Delta \widehat{t}^h_n = \Delta t^h_n I_{\{ \pi ^h_n =0 \}} + h I_{\{ \pi ^h_n \le -1 \}} + h I_{\{ \pi ^h_n \ge 1 \}},\\ &{}\displaystyle \widehat{t}_0=0, \qquad \widehat{t}_n = \sum \limits _{k=0}^{n-1}\Delta \widehat{t}^h_k, \quad n\ge 1.\\ \end{array} \end{aligned}$$

(B.13)

The time scale is stretched out by h at the seeding and harvesting steps.

Definition B.3

The rescaled time process $\widehat{T}^h(\cdot )$ is the unique continuous nondecreasing process satisfying the following:

• (a) $\widehat{T}^h(0)=0$;

• (b) the derivative of $\widehat{T}^h(\cdot )$ is 1 on $(\widehat{t}^h_n, \widehat{t}^h_{n+1})$ if $\pi ^h_n=0$, i.e., n is a diffusion step;

• (c) the derivative of $\widehat{T}^h(\cdot )$ is 0 on $(\widehat{t}^h_n, \widehat{t}^h_{n+1})$ if $\pi ^h_n\ne 0$, i.e., n is a seeding step or a harvesting step.

Thus $\widehat{T}^h(\cdot )$ does not increase at the times t at which a harvesting step or a seeding step occurs. Define the rescaled and interpolated process $\widehat{X}^h(t)= \xi ^h(\widehat{T}^h(t))$ and likewise define $\widehat{Y}^h(\cdot )$, $\widehat{Z}^h(\cdot )$, $\widehat{B}^h(\cdot )$, $\widehat{M}^h(\cdot )$, and the filtration $\widehat{{\mathcal {F}}}^h(\cdot )$ similarly. It follows from (B.7) that

$$\begin{aligned} \widehat{X}^h(t)=x+\widehat{B}^h(t) + \widehat{M}^h(t) - \widehat{Y}^h(t) + \widehat{Z}^h(t). \end{aligned}$$

(B.14)

Using the same argument we used for (B.11) we obtain

$$\begin{aligned}&\widehat{X}^h(t) = x + \int _0^t b (\widehat{X}^h(s))d\widehat{T}^h(s) \nonumber \\&\quad + \int _0^t \sigma (\widehat{X}^h(s)) d \widehat{W}^h(s) - \widehat{Y}^h(t) + \widehat{Z}^h(t) + \widehat{\varepsilon }^h(t), \end{aligned}$$

(B.15)

with $\widehat{\varepsilon }^h(\cdot )$ is an $\widehat{{\mathcal {F}}}^h(\cdot )$-adapted process satisfying

$$\begin{aligned} \lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{{\mathbb {E}}}|\widehat{\varepsilon }^h(t)|=0 \quad \text {for any }0<T_0<\infty . \end{aligned}$$

(B.16)

1.3 Convergence

Using weak convergence methods, we can obtain the convergence of the algorithms. The proofs to the following results are essentially the same as those given by Jin et al. (2013), Tran and Yin (2016) and we therefore omit the details.

Theorem B.4

Suppose Assumptions 2.3 and B.1 hold. Let the approximating chain $\{X^h_n \}$ be constructed with transition probabilities defined in (B.2)–(B.3), $\big (X^h(\cdot ), W^h(\cdot ), Y^h(\cdot ), Z^h(\cdot )\big )$ be the continuous-time interpolation defined in (B.4)–(B.5), (B.9), and $\widehat{T}^h(\cdot )$ be the process from Definition B.3. Let $\widehat{X}^h(\cdot ), \widehat{W}^h(\cdot ), \widehat{Y}^h(\cdot ), \widehat{Z}^h(\cdot )$ be the corresponding rescaled processes and denote

$$\begin{aligned} \widehat{H}^h(\cdot )=\Big (\widehat{X}^h(\cdot ), \widehat{W}^h(\cdot ), \widehat{Y}^h(\cdot ), \widehat{Z}^h(\cdot ), \widehat{T}^h(\cdot )\Big ). \end{aligned}$$

Then the family of processes $(\widehat{H}^h)_{h>0}$ is tight. As a result, $(\widehat{H}^h)_{h>0}$ has a weakly convergent subsequence with limit

$$\begin{aligned} \widehat{H}(\cdot )=\Big (\widehat{X}(\cdot ), \widehat{W}(\cdot ), \widehat{Y}(\cdot ), \widehat{Z}(\cdot ), \widehat{T}(\cdot )\Big ). \end{aligned}$$

We proceed to characterize the limit process.

Theorem B.5

Suppose Assumptions 2.3 and B.1 hold. Let $\widehat{{\mathcal {F}}}(t)$ be the $\sigma$-algebra generated by

$$\begin{aligned} \{\widehat{X}(s), \widehat{W}(s), \widehat{Y}(s), \widehat{Z}(s), \widehat{T}(s):s \le t\}. \end{aligned}$$

Then the following assertions hold.

(a):

$\widehat{W}(t)$ is an $\widehat{{\mathcal {F}}}(t)$-martingale with quadratic variation process $\widehat{T}(t)I_d$.

(b):

$\widehat{Y}(\cdot )$, $\widehat{Z}(\cdot )$, and $\widehat{T}(\cdot )$ are nondecreasing and nonnegative.

(c):

The limit processes satisfy

$$\begin{aligned} \widehat{X}(t)=x +\int _0^t b(\widehat{X}(s))d\widehat{T}(s)+\int _0^t \sigma (\widehat{X}(s))d\widehat{W}(s)-\widehat{Y}(t)+\widehat{Z}(t), \quad t\ge 0.\nonumber \\ \end{aligned}$$

(B.17)

For $t<\infty$, define the inverse $R(t)= \inf \{s: \widehat{T}(s)>t\}$. For any process $\widehat{\nu }(\cdot )$, define the time-rescaled process $({\bar{\nu }}(t))$ by ${\bar{\nu }}(t)= \widehat{\nu }(R(t))$. Let ${\mathcal {{\bar{F}}}}(t)$ be the $\sigma$-algebra generated by $\{{{\bar{X}}}(s), {{\bar{W}}}(s), {{\bar{Y}}}(s), {{\bar{Z}}}(s), {{\bar{R}}}(s): s\le t\}$. Let $V^h(x)$ and $V^U(x)$ be value the functions defined in (2.16) and (2.21), respectively.

Theorem B.6

Suppose Assumptions 2.3 and B.1 hold. The following assertions are true.

(a):

${\bar{R}}$ is right continuous, nondecreasing, and ${\bar{R}}(t)\rightarrow \infty$ as $t \rightarrow \infty$ with probability 1.

(b):

${\bar{Y}}$ and ${\bar{Z}}$ are right-continuous, nondecreasing, nonnegative, and $\mathcal {{\bar{F}}}(t)$-adapted processes.

(c):

${\bar{W}}(\cdot )$ is a standard $\mathcal {{\bar{F}}}(t)$ adapted d dimensional Brownian motion, and

$$\begin{aligned} {{\bar{X}}}(t)=x +\int _0^t b({\bar{X}}(s))ds+\int _0^t \sigma ({\bar{X}}(s))d{\bar{W}}(s)-{\bar{Y}}(t)+\bar{Z}(t), \quad t\ge 0.\nonumber \\ \end{aligned}$$

(B.18)

(d):

For any $x\in [0, U]^d$, $V^h(x)\rightarrow V^U(x)$ as $h\rightarrow 0$.