1 Introduction

The development of an efficacious prophylactic vaccine that stimulates the cytotoxic T-lymphocyte (CTL) immune response represents the best hope for control of human immunodeficiency virus (HIV) (Currie et al. 2006; Klausner et al. 2003; McMichael and Hanke 2003). CD8 cytotoxic T lymphocytes (CTLs) constitute a major component of the immune response. When HIV virus enters the body, it targets the CD4+ T cells, which are referred to as “helper” T cells. These “helper” T cells can be considered as “messengers” or the command center of the immune system: they signal other immune cells that an invader (HIV virus) is to be fought. The immune cells (virus-specific CTLs) then respond to this message and lyse infected cells by recognizing viral peptide epitopes displayed on the cell surface and bound to class I major histocompatibility complex (MHC) molecules (Smith 2004). The purpose of HIV immunotherapy is then to retain high levels of CD4+ T cells and low levels of viral load, along with, maintaining a positive population of CTLs so as to ensure that if viral load does rebound, the immune system will be able to handle it. Identification of a human immunodominant T-cell epitope (an epitope is the part of a protein that’s recognized by the immune system) is integral to vaccine design and the optimization of assays for assessing vaccine efficacy for the immune system can then lyse the infected cells by recognizing this particular epitope (Currie et al. 2006). Different strategies for HIV vaccine design have been proposed. For example, the study (Rossio et al. 1998) proposed a method for inactivation viral particles with preservation of conformational and functional integrity of virion surface proteins. This study showed that such inactivation virions should provide a promising candidate vaccine antigen and a useful reagent for experimental probing. The work (Luo et al. 1998) constructed chimeric fusion proteins using HIV-2 gag precursor protein to provide potential HIV vaccine against various HIVs. Balb/C mice immunized by this fusion protein can induce CTL activity against V3 peptide-stimulated target cells.

Mathematical models describing the interaction between the immune response and the viral infection have been studied intensively; see Culshaw et al. (2004), Nowak and McLean (1991), and references therein. Nowak and McLean (1991) established a mathematical model of vaccination with escape mutants and concluded that the fraction of HIV mutants that must be recognized by the immune response must exceed a threshold, that is, 1−1/R, where R is the average number of escape mutants produced by any one virus strain. Meanwhile, the immunotherapy has a better chance of success if applied earlier. However, most of these studies assume the vaccine is administered continuously which, in clinical trials, is hard to implement due to patient tolerance and financial burden.

Modeling study of pulse vaccination has recently gained much attention due to some successful applications in the control of poliomyelitis and measles throughout Central and South America (De Quadros et al. 1991; Sabin 1991). Among other contributions, modeling studies show that pulse vaccination strategies can be distinguished from conventional ones in leading to disease eradication at relatively low cost and tolerable side effects (Agur et al. 1993; Shulgin et al. 1998). Impulsive differential equations arise very naturally from the description of such vaccine program. For example, Smith and Schwartz (2008) modeled a CTL vaccine with fixed pulsing at regular intervals and concluded that the number of infected CD4+ T cells will be driven toward zero, if the vaccine is sufficiently strong or is given sufficiently often. This study, however, is based on an oversimplified assumption about the dynamics of the CD4+ T cells and the virus, in particular, that the infected CD4+ T cells are assumed to be produced at a constant rate. This assumption becomes unreasonable in the earliest or latest stages of infection when the amount of free virus is not constant. The results obtained by Smith and Schwartz (2008) under the constant production rate assumption then may be too strong to be realized, because too big dosages and/or too frequent vaccinations will cause serious side effects and/or toxicities to the individual. We extend the model by Smith and Schwartz (2008) to the classical HIV viral dynamical models consisting of both the wild-type strain and the mutant strain with CTL vaccine. We assume that cells infected with the mutant strain can be partially controlled by CTLs, but this (partial) control is less efficient for the wild-type strain based on the fact that HIV viruses can change their recognition sequence either via mutation or antigenic drift to escape CTLs of the immune responses, as mentioned in Nowak and McLean (1991). In our study, we consider not only fixed vaccination intervals as in Smith and Schwartz (2008), but varying vaccination intervals, which is motivated by the fact that, in clinical trials, a schedule to obtain the optimal therapeutic results may be the ones with fluctuating vaccination dates and/or vaccine dosages.

A goal of our work is then to study the threshold viral dynamics of the impulsive periodic system to determine the critical vaccination level (a combination of both vaccination frequency and dosage) to eradicate the viruses. We also examine the interactions of the two strains and the effects of escape mutants on the vaccine treatment. The second goal of our work is to consider the vaccine optimal treatment problem by formulating it as an impulsive optimal control problem in which the benefit based on levels of healthy CD4+ T cells and immune response cells are maximized. Our focus here is to design an optimal vaccination schedule including when and how much of the vaccine to administer so to have a prolonged and effective immune response.

The optimal control problem encountered in our framework is quite different from the conventional ones, as the control variables in our framework are not continuous in time due to instantaneous vaccination. This has been referred to as an optimal impulsive control problem. We utilize a generalized variational equation to compute the derivatives of the objective function with respect to the control variables and then apply some gradient based numerical methods to search for the maximum of the objective function. Note that classic within-host viral dynamical model is formulated on the basis of assumption of constant coefficients on average at the individual level. But in real-world situations, the viral dynamical coefficients may vary due to patient specificity and variability, which may affect the results in clinical practice. We will employ sensitivity analysis to examine the effects of patient variability on the critical vaccination levels.

The rest of this paper is organized as follows. In Sect. 2, the model with vaccination is presented as a hybrid system of ordinary differential equations and impulsive differential equations. The threshold viral dynamics is then given in Sect. 3 using the basic reproduction number as a threshold parameter. Further, the interactions of the two strains and the effects of escape mutants on the vaccine treatment are studied. In Sect. 4, the optimal impulsive control problem is formulated and solved using gradient-based numerical methods to determine when and how much of the vaccine to administer. Some remarks and discussions are included in the final section.

2 The Virus Dynamics Model with Vaccine

We modify the classical virus dynamics model by incorporating an immune response (Konrad et al. 2011; Lou et al. 2011; Rong et al. 2007) with fixed or fluctuating vaccination times and dosages. In what follows, the susceptible CD4+ T cells are denoted by T. We consider two virus strains, the wild-type strain V w and the mutant strain V r . CD4+ T cells infected with wild-type and mutant strains are denoted by T w , and T r , respectively. The density of the specialized CTL cells is denoted by C. We assume that, during the course of reverse transcription of viral RNA into proviral DNA, cells infected by the wild-type strain mutate at rate p, which is strictly positive. We ignore mutations from the mutant strain back to the wild-type (Wodarz and Lloyd 2004). We further assume that mutant variants arise with probability q during the course of wild-type viral replication.

Susceptible T cells are produced from precursors with a constant rate λ and die with rate δ T . Susceptible T cells are infected by the wild-type stain and the mutant strain at rates β w and β r , respectively. N w and N r (burst sizes) are the total number of virus particles released by productively infected cells infected by the wild-type strain and mutant strain over the lifespan with the same mean 1/δ I . Virus strains V w and V r are assumed to be cleared with the same rate δ V . Infected cells are also lysed by the body’s defensive CTLs with the rate p w and p r . CTLs proliferate at the presence of the infected cells at rate α and die at rate δ C . Continuous vaccination is not a realistic way of administering a vaccine, so in practical terms only pulse vaccination makes sense. At vaccination times t i ,i=1,2,… , vaccination increases CTL cells by a amount c i ,i=1,2,… . The virus dynamics can then be modeled using the following impulsive differential equations:

for t≠t i , the virus dynamics is given by

$$ \everymath{\displaystyle} \left \{ \begin{array}{l} T'(t)=\lambda-\delta_TT-\beta_wTV_w-\beta_rTV_r, \\[3pt] T_w'(t)=(1-p)\beta_wTV_w-\delta_IT_w-p_wT_wC,\\[3pt] V_w'(t)=(1-q)N_w\delta_IT_w-\delta_VV_w,\\[3pt] T_r'(t)=p\beta_wTV_w+\beta_rTV_r-\delta_IT_r-p_rT_rC,\\[3pt] V_r'(t)=qN_w\delta_IT_w+N_r\delta_IT_r-\delta_VV_r,\\[3pt] C'(t)=\alpha(T_w+T_r)C-\delta_CC; \end{array} \right . $$
(1)

at t=t i , the CTL cells are given by

$$ C\bigl(t_i^+\bigr)=C(t_i)+c_i. \\[3pt] $$
(2)

Here, C(t i ) is the CTL concentration immediately before the vaccination, and \(C(t_{i}^{+})\) is the CTL concentration immediately after the vaccination. The model formulation is illustrated in Fig. 1.

Fig. 1
figure 1

Susceptible T cells are produced at a constant rate λ and die at rate δ T . T cells infected by the wild-type strain and the mutant strain produce new virus particles at respective rates N w and N r . Infected T cells die at rate δ I . CTL cells proliferate at the presence of infected T cells at rate α. Due to mutation, a proportion p of T cells infected by the wild type become mutant. Mutant variants arise with rate q during the course of wild-type viral replication. CTLs then kill productively infected T cells at rates p w and p r , respectively

Full size image

The mutant strain possesses a fitness cost resulting in a reduced infectivity, so we assume that β w >β r (Smith 2004; Wahl and Nowak 2000). The wild-type virus is assumed to have higher replication rate, that is, N w >N r (Wodarz and Lloyd 2004). During in vitro experiments, in which the ability of the mutants Gag p11C, Env TL9, Env p41A, and Pol p68A to bind to MHC class I molecule Mamu-A*01 and be recognized by epitope-specific CTLs were assessed, partial CTL responses against mutant viruses were observed (Barouch et al. 2002), which implies that p r >0. HIV viruses can change their recognition sequence either via mutation or antigenic drift to escape CTLs of the immune responses, which also implies p w >p r (Konrad et al. 2011).

3 Threshold Dynamics with Fixed Vaccination Intervals

When the vaccine is administered at regular time intervals with fixed dosage, that is, t i+1−t i =τ, \(c_{i}=\tilde{C}\), the hybrid system (1)–(2) admits a virus-free periodic solution P 0=(T 0,0,0,0,0,C ∗(t)), where T 0=λ/δ T , and

$$C^*(t)=\frac{\tilde{C}e^{-\delta_C(t-i\tau)}}{1-e^{-\delta_C\tau}}, \quad i\tau<t\leq(i+1)\tau. $$

Using the same argument as for Lemma 2.2 (Yang and Xiao 2012), by calculating the stroboscopic map of the impulsive periodic orbit and applying the Floquet theory, we can show that C ∗(t) is the globally asymptomatically stable periodic solution of the impulsive periodic system

$$\everymath{\displaystyle} \begin{array}{l} \frac{dC}{dt}=-\delta_CC,\quad t\neq i\tau, \\[13pt] C\bigl(i\tau^+\bigr)=C(i\tau)+\tilde{C},\quad t=i\tau. \end{array} $$

We now calculate the basic reproduction number for the impulsive system (1)–(2) by using the next infection operator for continuous periodic systems proposed in Bacaër (2007), Wang and Zhao (2008) and the methods developed for the piecewise continuous periodic system in Yang and Xiao (2010, 2012). Define two matrices at the virus-free periodic solution P 0

Let Y(t,s),t≥s be the evolution operator of the linear τ-periodic system

$$\frac{dy}{dt}=-G(t)y(t). $$

That is, for each \(s \in \mathbb{R}\), the 4×4 matrix Y(t,s) satisfies

$$\frac{dY(t,s)}{dt}=-G(t)Y(t,s),\quad \forall t\geq s,\ Y(s,s)=I, $$

where I is the 4×4 identity matrix.

Define the next infection operator L on C ω :

where C ω is defined as the ordered Banach space of all τ-periodic functions from \(\mathbb{R}\) to \(\mathbb{R}^{4}\), equipped with the maximum norm ∥.∥, and the positive cone \(C_{\omega}^{+}:= \{\phi\in C_{\omega}:\phi(t)\geq 0, \forall t\in \mathbb{R}\}\). We then define the basic reproduction number as the spectral radius of L, that is,

$$R_0:=\rho(L). $$

The results in Wang and Zhao (2008) and Yang and Xiao (2010, 2012) ensure that the defined basic reproduction number is a threshold parameter. Assume that Φ F−G (t) is the principle fundamental solution to the linear system x′=(F−G(t))x. Then we have

  1. (i)

    R 0=1 if and only if ρ(Φ F−G (τ))=1;

  2. (ii)

    R 0>1 if and only if ρ(Φ F−G (τ))>1;

  3. (iii)

    R 0<1 if and only if ρ(Φ F−G (τ))<1;

  4. (iv)

    The “virus-free” periodic solution P 0=(λ/δ T ,0,0,0,0,C ∗(t)) is asymptotically stable if R 0<1, and unstable if R 0>1.

The basic reproduction number for the hybrid system (1)–(2) can essentially be interpreted in terms of the basic reproduction numbers for the wild-type and the mutant strains. Namely, we partition the matrices as follows:

where

Then we define the basic reproduction number for the wild-type strain as \(R_{0}^{w}:=\rho(L_{w})\), where the next infection operator L w is defined as

Y w (t,s) is the evolution operator of the τ-periodic system y′(t)=G w (t)y(t).

Similarly, the basic reproduction number for the mutant strain is \(R_{0}^{r}:=\rho(L_{r})\), with the next infection operator L r defined as

Y r (t,s) is the evolution operator of the τ-periodic system y′(t)=G r (t)y(t).

It follows that \(\rho(\varPhi_{F-G}(\tau))= \max\{\rho(\varPhi_{F_{w}-G_{w}}(\tau)),\rho(\varPhi_{F_{r}-G_{r}}(\tau))\}\). Then R 0<1 if and only if \(R_{0}^{w}<1\) and \(R_{0}^{r}<1\), R 0>1 if and only if \(R_{0}^{w}>1\) or \(R_{0}^{r}>1\). From the algorithm in Wang and Zhao (2008) and Yang and Xiao (2010, 2012) by considering an auxiliary linear impulsive periodic system \(w'(t)=(-G(t)+\frac{F}{\chi})w\), then R 0 (similarly for \(R_{0}^{w}\) and \(R_{0}^{r}\)) is the solution of the polynomial ρ(U(τ,0,χ))=1, where U(τ,0,χ) is the evolution operator of this auxiliary system. Then we can further observe that

$$R_0=\max\bigl\{R_0^w,R_0^r \bigr\}. $$

From the aforementioned, it can be seen that R 0 and ρ(Φ F−G (τ)) (similarly for \(R_{0}^{w}\) and \(\rho(\varPhi_{F_{w}-G_{w}}(\tau))\), \(R_{0}^{r}\) and \(\rho(\varPhi_{F_{r}-G_{r}}(\tau))\)) can both serve as the threshold parameters for virus eradication. However, R 0=1 and ρ(Φ F−G (τ))=1 may correspond to different critical vaccine thresholds, and from the biological meaning of the basic reproduction number, R 0 gives the exact critical vaccine thresholds for virus eradication.

From the definition of the basic reproduction number, it can be observed that R 0 (similarly for \(R_{0}^{w}\) and \(R_{0}^{r}\)) is the function of the vaccine dosage \(\tilde{C}\) and vaccination interval τ. Numerical simulations (summarized in Fig. 2(A)) indicate that there exist a threshold vaccination interval (τ ∗) and a threshold vaccine dosage (\(\tilde{C}^{*}\)) such that if the vaccine is administered with dosage bigger than \(\tilde{C}^{*}\) and/or administered more frequently (τ<τ ∗), the basic reproduction number will fall down below unity. Apparently, the results obtained here are easier to be realized than that obtained by Smith and Schwartz (2008) where the vaccine should be sufficiently strong and/or be given sufficiently often and no explicit thresholds were obtained. The parameters for the virus dynamics model are taken from Konrad et al. (2011), Lou et al. (2011), Smith and Schwartz (2008) and references therein, which are listed in Table 1.

Fig. 2
figure 2

(A) The progression of the basic reproduction number with respect to the vaccination interval τ and vaccine dosage \(\tilde{C}\). (B) PRCC for the basic reproduction number R 0 when , τ=20 days. All the parameters came from Latin Hypercube sampling

Full size image
Table 1 Parameter Values. We have mentioned in Sect. 2 that the wild-type strain is more infectious and has a higher replication rate than the mutant strain, that is, β w >β r , and N w >N r . Here, we take β r =0.95β w and N r =0.8N w to reflect this fact
Full size table

In real-world situations, the viral dynamical coefficients as listed in Table 1 may vary due to patient variability, we then employ partial rank correlated coefficient (PRCC) (Marino et al. 2008) to see to which parameter the basic reproduction number is sensitive when parameters vary. Fix the vaccine dosage , vaccination interval τ=20 days. We chose a uniform distribution as in (Marino et al. 2008) for all input parameters with the mean value listed in Table 1. PRCC results in Fig. 2(B) show that the first six with most significant impact on R 0 are the natural death rate of the uninfected cells δ T , the source of the uninfected cells from precursors λ, the burst sizes N w and N r , and the rates of infection β w and β r . It is reasonable that both the infection rates β w and β r , and the burst sizes N w and N r play important roles for the infection. Larger λ and smaller δ T leads to more susceptible uninfected CD4+ T cells, which may result in larger R 0.

In the following, we show explicitly how the vaccine level, described implicitly by the basic reproduction number, affects the elimination and the persistence of the two virus strains.

3.1 Sufficient Vaccination

The following result shows that if the vaccine is at a high level, leading to the basic reproduction number below unity, then both strains can be cleared out.

Theorem 3.1

If R 0<1, then the “virus-free” periodic solution P 0=(λ/δ T ,0,0,0,0,C ∗(t)) of the hybrid system (1)–(2) is globally asymptotically stable.

Proof

We have already described how to obtain the local stability of the “virus-free” periodic solution P 0 when R 0<1 is satisfied, so it is sufficient to prove that P 0 is globally attractive if R 0<1.

By the first equation of system (1) and the nonnegativity of the solutions, we have T′(t)≤λ−δ T T, and C′(t)≥−δ C C, t≠t i . Then a comparison argument implies that there exist a time t 1>0 and a positive constant ϵ 1 such that T(t)≤λ/δ T +ϵ 1, t≥t 1. Since the periodic solution C ∗(t) of the CTLs is globally asymptomatically stable, there exist a time t 2=n 0 τ≥t 1 and a positive constant ϵ 2 such that C(t)≥C ∗(t)−ϵ 2 for t≥t 2.

Then the second to the fifth equations of system (1) yields for t≥t 2 that

$$ \everymath{\displaystyle} \begin{array}{l} T_w'(t)\leq(1-p)\beta_w(T_0+\epsilon_1)V_w-\delta_IT_w-p_wT_w(C^*(t)-\epsilon_2),\\[3pt] V_w'(t)=(1-q)N_w\delta_IT_w-\delta_VV_w,\\[3pt] T_r'(t)\leq p\beta_w(T_0+\epsilon_1)V_w+\beta_r(T_0+\epsilon_1)V_r-\delta_IT_r-p_rT_r(C^*(t)-\epsilon_2),\\[3pt] V_r'(t)=qN_w\delta_IT_w+N_r\delta_IT_r-\delta_VV_r.\\[3pt] \end{array} $$
(3)

Consider an auxiliary system

$$ \frac{dz(t)}{dt}=\bigl(F-G(t)+ M(\epsilon_1, \epsilon_2)\bigr)z(t), $$
(4)

where z=(z 1,z 2,z 3,z 4)T, and the matrix

$$M(\epsilon_1,\epsilon_2)= \left ( \begin{array}{c@{\quad}c@{\quad}c@{\quad}c} p_w\epsilon_2 & (1-p)\beta_w\epsilon_1 & 0& 0 \\[3pt] 0 & 0 & 0 & 0 \\[3pt] 0 & p\beta_w\epsilon_1 & p_r\epsilon_2 & \beta_r\epsilon_1 \\[3pt] 0 & 0 & 0 & 0 \\[3pt] \end{array} \right ) . $$

By Zhang and Zhao (2007) (Lemma 2.1), there exits a positive, τ-periodic function v(t), such that \(z(t)=e^{\mu_{1} t}v(t)\) is a solution of system (4), where \(\mu_{1}=\frac{1}{\tau}\ln\rho(\varPhi_{F-G+M(\epsilon_{1},\epsilon_{2})}(\tau))\). Since R 0<1, we have ρ(Φ F−G (τ))<1. Due to the continuity of \(\rho(\varPhi_{F-G+M(\epsilon_{1},\epsilon_{2})}(\tau))\), we can choose sufficiently small ϵ 1, and ϵ 2 such that \(\rho(\varPhi_{F-G+M(\epsilon_{1},\epsilon_{2})}(\tau))<1\), that is μ 1<0. Therefore, we have z(t)→0 as t→∞. For any nonnegative initial value (T w (t 2),V w (t 2),T r (t 2),V r (t 2))T of system (3), there exits a z ∗ large enough such that

$$\bigl(T_w(t_2),V_w(t_2),T_r(t_2),V_r(t_2) \bigr)^T\leq z^*\bigl(v_1(0),v_2(0),v_3(0),v_4(0) \bigr)^T. $$

Then a comparison argument yields that

Hence, we have T w (t)→0,V w (t)→0,T r (t)→0,V r (t)→0 as t→∞. By the first and last equations of system (1), we have T→λ/δ T and C(t)→C ∗(t) as t→∞. This completes the proof. □

Corollary 3.1

If the basic reproduction number for the wild-type virus is smaller than one, that is, \(R_{0}^{w}<1\), then the wild-type strain cannot persist.

Proof

From the structure of the hybrid system (1)–(2), we can see that the equations for the wild-type strain T w and V w are not influenced by the mutant strain variables T r and V r . Then we can consider the system involving only T w and V w , and apply the argument for Theorem 3.1 to obtain the result. □

Remark 3.1

Theorem 3.1 indicates that if the vaccine is at a high level (big dosage and/or small vaccination interval) such that the basic reproduction number R 0<1, then both strains can be eliminated. While Corollary 3.1 indicates that if the basic reproduction number for the wily-type strain \(R_{0}^{w}<1\), then the wily-type strain can be cleared out. However, this is not the case when the basic reproduction number for the mutant strain \(R_{0}^{r}<1\), as shall be shown in the next subsection.

If the vaccination is not sufficient enough, leading to the basic reproduction number for the wild-type strain and/or the mutant strain larger than unity, that is,

$$\left \{ \begin{array}{l} R_0^w\leq 1,\qquad R_0^r>1,\\[3pt] R_0^w>1,\qquad R_0^r>1,\\[3pt] R_0^w>1,\qquad R_0^r\leq 1,\\[3pt] \end{array} \right . $$

then for all of these cases, the wild-type strain and the mutant strain not only compete with each other but may benefit from each other, as will be shown in the next subsection.

3.2 Insufficient Vaccination

Under an insufficient vaccination schedule, which is unable to drive the basic reproduction numbers for either or both of the two strains below unity, we can see that at least the mutant strain will persist. We start with the case when the vaccine is at a level such that the basic reproduction number for the mutant strain is larger than one.

Theorem 3.2

If the basic reproduction number for the mutant strain \(R_{0}^{r}>1\), then there exists a positive constant ε such that for every given value \((T^{0}, T_{w}^{0}, V_{w}^{0},T_{r}^{0}, V_{r}^{0},C^{0})\in R_{+} \times R_{+}^{2} \times \mathrm{Int}(R_{+}^{2})\times R_{+}\), the solution of system (1)–(2) satisfies lim inf t→∞(T r (t),V r (t))>ε and system (1)–(2) admits at least one nontrivial periodic solution (T(t),0,0,T r (t),V r (t),C(t)).

Define

Let \(P:R^{6}_{+} \rightarrow R^{6}_{+} \) be the Poincaré map associated with system (1)–(2), satisfying P(x 0)=u(τ,x 0), ∀x 0∈X, u(t,x 0) is the unique solution of (1)–(2) with u(0,x 0)=x 0. It can be seen that

$$P^m\bigl(T^0,T_w^0,V_w^0,T_r^0,V_r^0,C^0 \bigr)=u(m\tau, \bigl(T^0,T_w^0,V_w^0,T_r^0,V_r^0,C^0 \bigr),\quad \forall m\geq 0. $$

Clearly, the fixed point of the Poincaré map P in X is M 1=(T 0,0,0,0,0,C ∗(0)).

Lemma 3.1

If the basic reproduction number of the mutant strain \(R_{0}^{r}>1\), then there exists a δ 0>0 such that for all x 0∈X 0 with ∥x 0−M 1∥≤δ 0, where \(x^{0}=(T^{0},T_{w}^{0},V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in X_{0}\), we have

$$ \limsup_{m\rightarrow \infty}d\bigl(P^{m} \bigl(x^{0}\bigr),M_{1}\bigr)\geq \delta_{0}. $$
(5)

We defer the proof of this lemma to the Appendix. With this lemma, we can now give the proof of Theorem 3.2.

Proof of Theorem 3.2

We will show that P is uniformly persistent with respect to (X 0,∂X 0). It is easy to see from system (1)–(2) that X and X 0 are positively invariant. Moreover, ∂X 0 is a relatively closed set in X. Due to the uniform and ultimate boundedness of the solutions of system (1)–(2) (refer to Yang and Xiao 2012), the Poincaré map P admits a global attractor in X.

Set

We now show that

$$ M_{\partial}=\bigl\{(T,0,0,0,0,C):T\geq 0, C\geq 0\bigr\}. $$
(6)

Consider a given \((T^{0},T_{w}^{0},V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in \partial X_{0}\backslash \{(T,0,0,0,0,C):T\geq 0, C\geq 0\}\). We take \(T_{r}^{0}=0, T_{w}^{0},V_{w}^{0},V_{r}^{0}\) nonzero as an example. By the fourth equation of system (1), we can see that

$$T'_r(0)=p\beta_wT(0)V_w(0)+ \beta_rT(0)V_r(0)>0. $$

This implies that for t>0 sufficiently small, (T,T w ,V w ,T r ,V r ,C)∉∂X 0 and (T,T w ,V w ,T r ,V r ,C)∉M ∂ . Then the converse proposition indicates that (6) holds.

Clearly, there is exactly one fixed point of P in M ∂ , which is M 1=(T 0,0,0,0,0,C ∗(0)). Lemma 3.1 implies that M 1 is isolated in X and W s(M 1)∩X 0=∅. Clearly, each orbit in M ∂ converges to M 1, and M 1 is acyclic in M ∂ . By Theorem 1.3.1 in Zhao (2003), P is uniformly persistent with respect to (X 0,∂X 0). By Theorem 3.1.1 in Zhao (2003), the solutions of (1)–(2) are uniformly persistent with respect to (X 0,∂X 0). If one (or more) of \((T_{r}^{0},V_{r}^{0})\) is (or are) larger than 0, it can be easily obtained that T r (t)>0,V r (t)>0,∀t>0. Hence, for a given initial value \((T^{0}, T_{w}^{0}, V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in R_{+} \times R_{+}^{2} \times \mathop{\mathrm{Int}}(R_{+}^{2})\times R_{+}\), lim inf t→∞(T r (t),V r (t))>ε.

Furthermore, Theorem 1.3.6 in Zhao (2003) implies that P has a fixed point x ∗∈X 0. Thus, the solution of system (1)–(2) through x ∗ is a nontrivial periodic solution with T r (t)>0,V r (t)>0. This completes the proof. □

Corollary 3.2

If the vaccine is at a low level such that \(R_{0}^{w}>1\), and \(R_{0}^{r}>1\), then there exists a positive constant ε such that for any given initial value \((T^{0}, T_{w}^{0}, V_{w}^{0},T_{r}^{0}, V_{r}^{0},C^{0})\in R_{+} \times \mathop{\mathrm{Int}}(R_{+}^{4})\times R_{+}\), the solution of system (1)–(2) satisfies lim inf t→∞(T w (t),V w (t),T r (t),V r (t))>ε and system (1)–(2) admits at least one positive periodic solution (T(t),T w (t),V w (t),T r (t),V r (t),C(t)).

The proof of Corollary 3.2 is similar to that for Theorem 3.2, using X 0={(T,T w ,V w ,T r ,V r ,C)∈X:T w >0,V w >0,T r >0,V r >0}.

Remark 3.2

Theorem 3.2 and Corollaries 3.1, 3.2 indicate that if the basic reproduction numbers for the wild-type strain \(R_{0}^{w}<1\) and for the mutant strain \(R_{0}^{r}>1\), then the mutant strain will out-compete the wild-type strain and dominate. These results also show that both strains persist if \(R_{0}^{w}>1\), and \(R_{0}^{r}>1\).

Notice that if the basic reproduction number for the wild-type strain is less than one, then the wild-type strain is cleared out. However, this is not the case when the basic reproduction number for the mutant strain is less than one. If the vaccine results in \(R_{0}^{w}>1\), the next result shows that no matter what the size of \(R_{0}^{r}\) is, both of the two strains can persist.

Theorem 3.3

If the basic reproduction number for the wild-type strain \(R_{0}^{w}>1\), then there exists a positive constant ϱ such that for every given initial value \((T^{0}, T_{w}^{0}, V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in R_{+} \times \mathop{\mathrm{Int}}(R_{+}^{2})\times R_{+}^{2}\times R_{+}\), the solution of system (1)–(2) satisfies

$$\liminf_{t\rightarrow\infty}\bigl(T_w(t),V_w(t),T_r(t),V_r(t) \bigr)>\varrho, $$

and system (1)–(2) admits at least one positive periodic solution \((T^{*}(t),T_{w}^{*}(t),V_{w}^{*}(t), T_{r}^{*}(t),V_{r}^{*}(t),C_{*}(t))\).

Proof

Define

Let P be the Poincaré map associated with system (1)–(2) as aforedefined, and set

We now show that

$$ \bar{M}_{\partial}=\bigl\{(T,0,0,T_r,V_r,C):T \geq 0, T_r\geq0, V_r\geq0, C\geq 0\bigr\}. $$
(7)

We consider any \((T^{0},T_{w}^{0},V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in \partial Y_{0}\backslash \{(T,0,0,T_{r},V_{r},C):T\geq 0, T_{r}\geq0, V_{r}\geq0, C\geq 0\}\), but focus on the case \(T_{w}^{0}=0, V_{w}^{0}\neq 0\) as an example. By the second equation of system (1) we can see that

$$T'_w(0)=(1-p)\beta_wT(0)V_w(0)>0. $$

This implies that for t>0 sufficiently small, (T,T w ,V w ,T r ,V r ,C)∉∂Y 0, then \((T, T_{w}, V_{w}, T_{r},V_{r},C)\notin \bar{M}_{\partial}\). Then the converse proposition indicates that (7) holds.

For the case when \(R_{0}^{r}>1\), the results follow from Corollary 3.2. We now concentrate on the case when \(R_{0}^{r}<1\). We show that under the condition \(R_{0}^{r}<1\), the solutions of the hybrid system (1)–(2) initiating from \(\bar{M}_{\partial}\) converge to (T 0,0,0,0,0,C ∗(t)). For the initial values from \(\bar{M}_{\partial}\), it follows that T w (t)=V w (t)=0, then the hybrid system (1)–(2) can be reduced to the following:

for t≠t i :

$$ \everymath{\displaystyle} \left \{ \begin{array}{l} T'(t)=\lambda-\delta_TT-\beta_rTV_r, \\[3pt] T_r'(t)=\beta_rTV_r-\delta_IT_r-p_rT_rC,\\[3pt] V_r'(t)=N_r\delta_IT_r-\delta_VV_r,\\[3pt] C'(t)=\alpha T_rC-\delta_CC, \end{array} \right . $$
(8)

at t=t i :

$$ C\bigl(t_i^+\bigr)=C(t_i)+c_i. $$
(9)

It can be deduced, by applying the method in Theorem 3.1, that if \(R_{0}^{r}<1\) then the solutions of system (8)–(9) satisfy

$$\bigl(T(t),T_r(t),V_r(t)\bigr)\rightarrow \bigl(T_0,0,0,C^*(t)\bigr), $$

which then indicates that each orbit in \(\bar{M}_{\partial}\) converges to M 1.

Then using similar methods as that in Theorem 3.2, we can obtain that there exists a positive constant ϱ 1 such that for a given initial value \((T^{0}, T_{w}^{0}, V_{w}^{0},T_{r}^{0},V_{r}^{0},C^{0})\in R_{+} \times \mathop{\mathrm{Int}}(R_{+}^{2})\times R_{+}^{2}\times R_{+}\), the solution of system (1)–(2) satisfies

$$\liminf_{t\rightarrow\infty}\bigl(T_w(t),V_w(t)\bigr)> \varrho_1. $$

Due to the structure of system (1), we can see that T w (t)>0 and V w (t)>0 indicates T r (t)>0, and V r (t)>0. Then it follows that there exists ϱ 2>0 such that

$$\liminf_{t\rightarrow\infty}\bigl(T_r(t),V_r(t)\bigr)> \varrho_2. $$

Let ϱ=max{ϱ 1,ϱ 2}, we obtain the coexistence for both strains. Also, Theorem 1.3.6 in Zhao (2003) ensures the existence of a positive periodic solution. □

We summarize in Table 2 the results obtained,indicating to see elaborately the out-competition and/or the benefit of the two strains with different levels of vaccination leading to different basic reproduction numbers \(R_{0}^{w}\) and \(R_{0}^{r}\).

Table 2 The progression of the two strains with respect to different basic reproduction numbers \(R_{0}^{w}\) and \(R_{0}^{r}\)
Full size table

We present in Fig. 3 the temporal evolution of the two virus strains at different levels of vaccination which results in different sizes of \(R_{0}^{w}\) and \(R_{0}^{r}\). It can be seen that high vaccination level yields eradication of the two strains, while intermediate and low vaccine levels may result in the competition or even cooperative of the two strains.

Fig. 3
figure 3

Simulated time courses of the viruses with sufficient and insufficient levels of vaccine. Parameter values are shown in Table 1. (A) At high levels of vaccine with \(R_{0}^{w}=0.8586<1\) and \(R_{0}^{r}=0.7618<1\), both strains can be cleared out. (B) At intermediate levels of vaccine with \(R_{0}^{w}=0.4288<1\) and \(R_{0}^{r}=1.5231>1\), the mutant strain out-competes and dominates. (C) At intermediate levels of vaccine with \(R_{0}^{w}=1.3534>1\) and \(R_{0}^{r}=0.9793<1\), the wild-type strain benefits the mutant strain, such that the two strains can persist. (D) At low levels of vaccine with \(R_{0}^{w}=1.6289>1\) and \(R_{0}^{r}=1.1722>1\), both strains persist

Full size image

4 Optimal Timing and Dosage

In clinical trials, optimal vaccination timing and vaccine dosage are crucial for the best therapeutic results while avoiding the toxicities as much as possible. To describe this optimal control problem, we reformulate the vaccination program as the sum of some Dirac-delta functions to represent a dosage c i vaccinated at time t i , with N as the total number of vaccinations in the treatment period, that is,

$$u(t)=\sum_{i=1}^Nc_i\delta(t-t_i). $$

As such, we can rewrite the hybrid system (1)–(2) as

$$ \everymath{\displaystyle} \begin{array}{l} T'(t)=\lambda-\delta_TT-\beta_wTV_w-\beta_rTV_r, \\[3pt] T_w'(t)=(1-p)\beta_wTV_w-\delta_IT_w-p_wT_wC,\\[3pt] V_w'(t)=(1-q)N_w\delta_IT_w-\delta_VV_w,\\[3pt] T_r'(t)=p\beta_wTV_w+\beta_rTV_r-\delta_IT_r-p_rT_rC,\\[3pt] V_r'(t)=qN_w\delta_IT_w+N_r\delta_IT_r-\delta_VV_r,\\[3pt] C'(t)=\alpha(T_w+T_r)C-\delta_CC+\mathbf{u(t)}.\\[3pt] \end{array} $$
(10)

There have been intensive studies about optimal controls for HIV chemotherapy; see Culshaw et al. (2004), Fister et al. (1998), Kirschner et al. (1997), Wein et al. (1997), and references therein. In these studies, the control function u(t) is continuous in time, and the control problem is referred to as a continuous control problem. For continuous control problems, there are some standard theories, notably the Pontryagin maximum principle (PMP) (Fleming and Rishel 1975; Pontryagin et al. 1986). This principle involves solving the associated Hamiltonian system, so the optimal control u ∗ in terms of the state variables and the adjoint variables can be obtained. The optimal control u ∗ can then be obtained by solving a two-point initial-boundary value problem numerically.

Here, we consider a special class of optimal control problems in which the dynamical system (10) involves a finite number of switching times t i , together with a state jump c i ,i=1,…,N, at each of these switching times. These optimal control problems are referred to as optimal impulsive control problems. Optimal control problems for general impulsive systems are difficult to study (Bensoussan and Lions 1984), though some progress has been achieved for the existence of solutions (Barles 1985; Li and Yong 1995 and references therein). A computational method was developed in Liu et al. (1998) to solve this class of impulsive control problems by transforming them into optimal parameter selection problems. We will use this method, but first let us reformulate the optimal impulsive control problem.

Consider a vaccine procedure according to the schedule

$$S=(\xi,c)=\Biggl\{(t_i,c_i):i=1,\ldots,N,0\leq t_1\leq t_2\leq\cdots\leq t_N\leq t_\mathrm{final},\sum_{i=1}^Nc_i \leq C_\mathrm{tot}\Biggr\}, $$

in a treatment period of time t final. Let Γ be the space of the schedules. We place a constraint on the cumulative vaccine dosage C tot, and hence the optimal control problem can be defined as follows.

Problem (P)

Subject to the system (10) with the initial condition \((T^{0},T_{w}^{0},,V_{w}^{0}, T_{r}^{0}V_{r}^{0},C^{0})\), find a schedule S=(ξ,z)∈Γ such that the objective function, the total CD4+ T cells and CTLs during the treatment period

$$ J(\xi,c)=J(t_1,t_2, \ldots,t_N,c_1,c_2,\ldots,c_N)= \int_0^{t_\mathrm{final}} \bigl(T(t)+C(t) \bigr)\,dt $$
(11)

can be maximized.

We seek an optimal pair \((\xi^{*},c^{*})=(t_{1}^{*},t_{2}^{*},\ldots,t_{N}^{*},c_{1}^{*},c_{2}^{*},\ldots,c_{N}^{*})\), such that

$$J\bigl(\xi^*,c^*\bigr)=\max\bigl\{J(\xi,c)\mid (\xi,c)\in\varGamma\bigr\}. $$

The optimal control problem (P) is a finite dimensional optimization problem. In fact, the control variable space Γ can be clearly parameterized by a subset of R 2N. Problem (P) cannot be solved directly by using available optimal control or optimization techniques. The states T(t) and C(t) are functions of t, t i , and c i since they depend on the vaccination times t i and vaccine dosage c i . Note that, in general, a close form describes the dependence of the objective function on its variables is not available. Our goal is to compute the derivatives of the objective function with respect to (t i ,c i ), i=1,…,N, and then to apply gradient based numerical methods to search for the maximum of the objective function.

We first show the existence of the optimal solution for Problem (P).

4.1 Existence of an Optimal Solution for Problem (P)

To use the result, Theorem III.4.1 from Fleming and Rishel (1975) about the existence of an optimal solution, we must check the following properties:

  1. (i)

    The control variable space Γ is compact;

  2. (ii)

    The function S→J(S) is continuous.

The space Γ is obviously compact, the continuity of the objective function J(S) with respect to the control space S can be obtained by transforming the model (10) to the equivalent system with the control variables (t i ,c i ),i=1,…,N as the parameters, suggested by Liu et al. (1998).

Let X=(T,T w ,V w ,T r ,V r ,C)T and rewrite system (10) as

$$ \everymath{\displaystyle} \begin{array}{l} X'(t)=F(X)+\sum_{i=1}^Nc_i\delta(t-t_i)\mathbf{e}_6, \\[17pt] X(0)=X_0=\bigl(T^0,T_w^0,V_w^0,T_r^0,V_r^0,C^0\bigr), \end{array} $$
(12)

where F:R 6→R 6 and e 6=(0,0,0,0,0,1)T. For i=1,…,N+1, define

$$ Y_i=X\bigl(t_{i-1}+(t_i-t_{i-1})s \bigr), $$
(13)

with t 0=0 and t N+1=t final. Using the chain rule, we express (12) in terms of the equivalent system (for Y ij , i=1,…,N+1, j=1,…,6, the index i represents the time interval (t i−1,t i ), j represents the equation for the jth state in system (12)):

for i=1,…,N+1,

$$ \everymath{\displaystyle} \begin{array} {l} Y_{i1}^{'}(s)=(t_i-t_{i-1})[ \lambda-\delta_TY_{i1}-\beta_wY_{i1}Y_{i3}- \beta_rY_{i1}Y_{i5}], \\[3pt] Y_{i2}^{'}(s)=(t_i-t_{i-1})\bigl[(1-p) \beta_wY_{i1}Y_{i3}-\delta_IY_{i2}-p_wY_{i2}Y_{i6} \bigr], \\[3pt] Y_{i3}^{'}(s)=(t_i-t_{i-1}) \bigl[(1-q)N_w\delta_IY_{i2}- \delta_VY_{i3}\bigr], \\[3pt] Y_{i4}^{'}(s)=(t_i-t_{i-1})[p \beta_wY_{i1}Y_{i3}+\beta_rY_{i1}Y_{i5}- \delta_IY_{i4}-p_rY_{i4}Y_{i6}], \\[3pt] Y_{i5}^{'}(s)=(t_i-t_{i-1})[qN_w \delta_IY_{i2}+N_r\delta_IY_{i4}- \delta_VY_{i5}], \\[3pt] Y_{i6}^{'}(s)=(t_i-t_{i-1})\bigl[ \alpha(Y_{i2}+Y_{i4})Y_{i6}-\delta_CY_{i6} \bigr]; \end{array} $$
(14)

with the initial conditions:

$$ \everymath{\displaystyle} \begin{array}{l} \mbox{for}\ i=1, \\[3pt] Y_{11}(0)=T^0,\qquad Y_{12}(0)=T_w^0,\qquad Y_{13}(0)=V_w^0,\qquad Y_{14}(0)=T_r^0,\\[3pt] Y_{15}(0)=V_r^0,\qquad Y_{16}(0)=C^0; \\[3pt] \mbox{for}\ i=2,\ldots,N+1,\\[3pt] Y_{ij}(0)=Y_{(i-1)j}(1),\quad j=1,2,3,4,5,\qquad Y_{i6}(0)=Y_{(i-1)6}(1)+c_{i-1}. \end{array} $$
(15)

Applying the transform t=t i−1+(t i −t i−1)s,s∈[0,1] and (13), we obtain

Then we have

(16)

It can be easily deduced that the continuity of the objective function (11) with respect to the vaccination time t i and vaccine dosage c i is equivalent to the continuity of the transformed function (16) with respect to t i and c i . Due to the continuity of the solution of system (14) with respect to the initial data (15) and parameters t i , the continuity of Y i1(s) and Y i6(s) with respect to t i and c i can be easily obtained. This gives the continuity of the objective function (11) with respect to t i and c i , which implies the existence of an optimal solution for Problem (P).

Liu et al. (1998) presented gradient formulas for the transformed problem (16) with respect to t i and c i , and then used the optimal control software package MISER 3.2 (Jennings et al. 1991) to solve the control problem. Here, we resort to the variational equation (Cappuccio et al. 2007; Castiglione and Piccoli 2006, 2007; Piccoli and Castiglione 2006) to calculate the gradient of the objective function (11) with respect to the variables t i and c i , and then use a gradient-based algorithm to obtain the optimal schedule.

4.2 Numerical Algorithms and Results

To better understand the variational equation method, we first present the definition of the variational equation. Rewrite the state system (12) as

$$\frac{dX}{dt}=F(X)+u\mathbf{e}_6=G(X,u). $$

Given a candidate optimal control u ∗ and corresponding trajectory X ∗, a time τ and another control ω∈Γ, a needle variation is a family of controls u ϵ obtained replacing u ∗ with ω on the interval [τ−ϵ,τ]. A needle variation gives rises to a variation v of the trajectory, for t≥τ, the variational equation with the initial conditions gives

$$\left \{\begin{array}{l} \frac{dv}{dt}=D_XG(X^*(t),u^*(t)).v(t),\\[3pt] v(\tau)=G(X^*(\tau),\omega)-G(X^*(\tau),u^*(\tau)). \end{array} \right . $$

For our special control system with the controls occurring in some instantaneous times, we can now present a more precise result.

Consider a family of controls u ϵ corresponding to a single vaccine administration procedure that happens at time \(t_{i}^{\epsilon}=t_{i}+\epsilon\). The family u ϵ gives rise to a trajectory variation characterized by the following lemma (similar to Proposition 1 in Castiglione and Piccoli 2006):

Lemma 4.1

Let u ϵ be a family of controls corresponding to a single vaccine administration procedure that happens at time \(t_{i}^{\epsilon}=t_{i}+\epsilon\). The corresponding variation for t≥t i is given by

$$\left \{\begin{array}{l} \frac{dv}{dt}=D_XF(X^*(t)).v(t),\\[3pt] v(t_i)=F(X^*(t_i))-F(X^*(t_i)+c_i\mathbf{e}_6). \end{array} \right . $$

The objective function (11) can be reduced to a new one depending only on the final states by introducing a new variable x 7(t). We set

Then the objective function (11) is reduced to

$$ x_7(t_\mathrm{final})=J(t_1, \ldots,t_N,c_1,\ldots,c_N)=\int _0^{t_\mathrm{final}} \bigl(T(t)+C(t) \bigr)\,dt. $$
(17)

For every vaccination schedule S∈Γ, let X S (t) be the corresponding trajectory with initial conditions as that in (12).

The gradient of the objective function (17) with respect to the vaccination times t i and vaccine dosage c i are given in the next propositions.

Proposition 4.1

Consider a vaccination schedule at time t i with vaccine dosage c i . For t≥t i , the vector (v 1(t),…,v 6(t),w(t)) with v=(v 1,…,v 6) solves the following equations:

$$ \everymath{\displaystyle} \begin{array} {l} \frac{dv}{dt}= \nabla_XF\bigl(X_S(t)\bigr).v(t), \\[11pt] \frac{dw}{dt}=\nabla_XL\bigl(X_S(t)\bigr).v(t), \\[11pt] v(t_i)=F\bigl(X_S(t_i)\bigr)-F \bigl(X_S(t_i)+c_i\mathbf{e}_6 \bigr), \\[7pt] w(t_i)=L\bigl(X_S(t_i)\bigr)-L \bigl(X_S(t_i)+c_i\mathbf{e}_6 \bigr), \end{array} $$
(18)

and

$$ w(t_\mathrm{final})=\frac{\partial x_7(t_\mathrm{final})}{\partial t_i}=\frac{\partial J}{\partial t_i}. $$

Proposition 4.2

Consider a vaccination schedule at time t i with vaccine dosage c i . For t≥t i , the vector (V 1(t),…,V 6(t),W(t)) with V=(V 1,…,V 6) solves the following equations:

$$ \everymath{\displaystyle} \begin{array} {l} \frac{dV}{dt}= \nabla_XF\bigl(X_S(t)\bigr).V(t), \\[11pt] \frac{dW}{dt}=\nabla_XL\bigl(X_S(t)\bigr).V(t), \\[7pt] V(t_i)=\mathbf{e}_6, \\[7pt] W(t_i)=0, \end{array} $$
(19)

and

$$ W(t_\mathrm{final})=\frac{\partial x_7(t_\mathrm{final})}{\partial c_i}=\frac{\partial J}{\partial c_i}. $$

The proofs of the above propositions are easily obtained using Taylor expansions of the involved quantities. Meanwhile, Propositions 4.1 and 4.2 give the basic ingredients for the numerical solution of the optimal control problem (P). We can apply the steepest descent or other optimization methods, which consists of the following procedures.

Optimization Algorithm

  1. Step 0.

    Fix the treatment period t final, an initial value X 0 for system (10), the number N of vaccine administrations and an initial schedule \((t_{1}^{0},t_{2}^{0},\ldots,t_{N}^{0},c_{1}^{0}, c_{2}^{0},\ldots,c_{N}^{0})\).

  2. Step 1.

    Solve the system (10) with the initial value X 0 and the initial schedule via the fourth-order Runge–Kutta method. At the same time, for every i=1,…,N, solve Eqs. (18) and (19).

  3. Step 2.

    Compute the gradient of the objective function with respect to t i and c i via the Propositions 4.1 and 4.2.

  4. Step 3.

    Update the schedule by steepest decent method, i.e., for i=1,…,N,

    $$t_i^{n+1}=t_i^n-h_t. \frac{\partial J}{\partial t_i},\qquad c_i^{n+1}=c_i^n-h_c. \frac{\partial J}{\partial c_i}, $$

    where n is the iterations, and h t and h c are small positive parameters. Go to Step 0.

In the following, we normalize the treatment period t final to be one year. The initial conditions for the state system (10) are set to be (300, 1, 1000, 0.001, 0.001, 2.5 mm−3). The parameters used are listed in Table 1. We start with a random vaccination schedule and the initial vaccine dosage is set to c i =35 cells/μL (the value used in Smith and Schwartz 2008). The schedule varies in both timing and dosage. We examine scenarios where four, five, and six vaccine injections are administered (N=4,5,6). We run the optimization schema Step 0–Step 3 for 1400 steps (no special stoping criteria is used here). In all of these three cases (N=4,5,6), we observe that the optimization algorithm tends to converge to three times of vaccine injections (We obtained the same conclusions even for a larger number N of vaccine injections; results not shown here). This is illustrated in Fig. 4 where we take the number of vaccine administration as N=6. The numbers 1,2,3,4,5,6 in Fig. 4 represent the first, the second to the sixth vaccine administrations at the very beginning, respectively. From Fig. 4, we notice that the schedule is characterized as follows: some vaccinations are glued together, such as the first and the second administrations, the third and the fourth administrations, the fifth and the sixth administrations, which results in six vaccinations finally converged to three ones (seen in Fig. 4(A), further illustrated elaborately in Fig. 4(B)-the zooming of Fig. 4(A)). Further noticed is that for the final three vaccine administrations, the second vaccination is administered soon after the first one, and the third vaccination is administered a long time later after the second one, seen in Fig. 4(A). Moreover, as the number of vaccine administrations of the initial six converges to three, what follows is that the six dosages increase from the same value c i =35 cells/μL, i=1,…,6, and end up converging to three different dosages, with the first quantity increasing to a high level while the other two dosages are more or less the same low level, seen in Fig. 4(C). It is intuitively reasonable to administer the first vaccine with bigger dosage (Fig. 4(C)) to strongly simulate the immune response to fight the disease, and then administer the second vaccine soon after the first one (Fig. 4(A)) to further strengthen the immune response. The third vaccination can be administered a long time later after the second one with a small dosage in case that if the viruses do rebound, the immune response is able to handle it. If the vaccine is administered one or two times (N=1,2), numerical simulations indicate that the immune response cannot be effectively boosted resulting in that the viruses cannot be effectively controlled. The simulations support that the optimization algorithm is stable and efficient. We present in Fig. 5 the evolution of the viruses when no vaccine is administered (the dashed line) in comparison with when the optimal schedule obtained in Fig. 4 is administered (solid line). It can be seen clearly that when the optimal schedule is administered, the viruses can be driven to a very low level and even eradicated.

Fig. 4
figure 4

Evolution of the optimal schedule during the optimization. The numbers 1,2,…,6 correspond to the first, the second, … , the sixth vaccination. (A) Initial six time vaccinations converges to a three time vaccination program. (B) Zooming of some optimization dynamics in (A). (C) Evolution of the vaccine dosage

Full size image
Fig. 5
figure 5

The comparison of the virus dynamics between the optimal vaccination schedule (solid line) and no treatment (dashed line). (A) The wild-type virus. (B) The mutant virus. The inset plot in B shows the optimal CTL vaccine. The optimal vaccination schedule is the one shown in Fig. 4

Full size image

5 Conclusions and Discussions

A vaccination program was proposed in Konrad et al. (2011), where the critical vaccine threshold for the eradication of the virus, and the conditions for the out-compete of the mutant strain (that is, the mutant strain dominated, while the wild-type strain was eradicated) were obtained. However, this study simplified the dynamics of the CTLs by viewing it as a parameter independent of time, which essentially results in an autonomous and continuous system of ordinary differential equations. Our model and analysis are based on not only the autonomous and continuous system (1) for the virus dynamics but also on the discrete perturbation due to the vaccine administration (2). The system becomes a hybrid impulsive system.

Our main results show that a CTL vaccine administered with fixed time intervals can theoretically eradicate both the wild-type and the mutant virions if the vaccine results in a high level of CTL concentration such that the basic reproduction number R 0 (the maximum of the basic reproduction numbers for the wild-type strain \(R_{0}^{w}\) and the mutant strain \(R_{0}^{r}\), namely, \(R_{0}=\max\{R_{0}^{w},R_{0}^{r}\}\)) of the hybrid system fall down below unity. If the basic reproduction number for the wild-type strain is less than unity, then the wild-type virus can be eradicated. However, this is not the case when the basic reproduction number for the mutant strain is less than unity, for if the basic reproduction number for the wild-type strain is larger than unity, the mutant strain will benefit from the wild-type strain such that both strains can persist. If either or both of the basic reproduction numbers for the two strains larger than unity, at least the mutant strain can persist.

It should be mentioned that patient specificity and variability inevitably induce variation in coefficients of the viral dynamic system, which may affect our main results. We then employed partial rank correlated coefficient (PRCC) (Marino et al. 2008) to investigate the sensitivity analysis. We obtained that the basic reproduction numb R 0 is most sensitive to variation in the natural death rate of the uninfected cells δ T (as shown in Fig. 2(B)). For the optimal vaccination design, repeatedly running the algorithm by varying the viral dynamical coefficients, we can obtain the similar results. That is, initial six vaccinations converged to final three vaccinations, but with different final levels of the vaccine dosage and vaccination timings.

In the virus dynamics model (1), we only consider the wild type strain and a mutant strain. However, in general this is reductive since HIV has such a mutation rate that “hundreds” of mutants are produced. Given multiple mutant strains we can similarly extend our model, define the basic reproduction number for the whole system and the corresponding basic reproduction numbers for different strains. The globally asymptotical stability of the “virus-free” periodic solution can be obtained by using similar analysis, and moreover, the optimal vaccination design can also be obtained. However, our model unfortunately cannot describe the stochastic mutation of HIV during replication, but one can resort to modeling methods with stochastic events developed by Christian and Rob (2008).

To obtain better therapeutic results and avoid toxicities, clinicians aim to design an optimal vaccination schedule consisting of the timing when to administer the vaccine and the amount of dosage to administer. To answer this question, we formulated an optimal control problem to maximize the benefits based on levels of healthy CD4+ T cells and the immune response cells. Unlike general standard optimal control problems with the control functions being continuous in time, which can be resorted to standard theories such as the Pontryagin Maximum Principle to solve the associated Hamilton system, the optimal control problem in our framework falls into the category of the so-called impulsive control, which is quite different from the continuous one. We obtained the gradient of the objective function with respect to the vaccination timing t i and the vaccine dosage c i via the solution of a generalized variational equation, which can be solved at the same time with the state system. The optimization algorithm was then implemented based on some gradient-based methods. A treatment period of one year is taken to be the optimization horizon. Numerical results indicate that the optimal schedule should consist of three times of vaccine administrations, with a big first dosage to strongly boost the immune responses, and then administer the second vaccine soon after the first one to further strengthen the immune responses, and concluded that the third vaccination administered a long time later after the second one with a small dosage. This is reasonable in clinical trails to ensure that if the viruses rebound, the immune response is able to handle it. The results seem to be quite satisfactory since the optimized schedule is always able to drive the virus to low levels or eradicate the virus, while maintaining a high level of healthy CD4+ T cells and immune responses.