Control problems and management policies in health systems: application to intensive care units

Abstract

The stochastic nature of both patient arrivals and lengths of stay leads inevitably to periodic bed shortages in healthcare units. Physicians are challenged to fit demand to service capacity. If all beds are occupied eligible patients are usually referred to another ward or hospital and scheduled surgeries may be cancelled. Lack of beds may also have consequences for patients, who may be discharged in advance when the number of occupied beds is so high as to compromise the medical care of new incoming patients. In this paper we deal with the problem of obtaining efficient bed-management policies. We introduce a queuing control problem in which neither the arrival rates nor the number of servers can be modified. Bed occupancy control is addressed by modifying the service time rates, to make them dependent on the state of the system. The objective functions are two quality-of-service components: to minimize patient rejections and to minimize the length of stay shortening. The first objective has a clear mathematical formulation: minimize the probability of rejecting a patient. The second objective admits several formulations. Four different expressions, all leading to nonlinear optimization problems, are proposed. The solutions of these optimization problems define different control policies. We obtain the analytical solutions by adopting Markov-type assumptions and comparing them in terms of the two quality-of-service components. We extend these results to the general case using optimization with simulation, and propose a way to simulate general length of stay distributions enabling the inclusion of state-dependent service rates.

1 Introduction

Intensive care units (ICU) are specialized sections of a hospital that provide care for critically ill patients requiring immediate attention. The type of equipment and clinicians needed to staff intensive care units makes them very costly to run and it is a challenge for hospital managers to fit demand to service capacity.

An ICU can be mathematically modelled as a queuing system, where the clients are the patients, the servers are the beds and there is no waiting room. Queuing theory has been widely used in the health care context, (see Lakshmi and Sivakumar (2013) for a review of its applications in health care management problems). Some studies have used queuing models to address management problems in various hospital departments [emergency department (ED), ICU, wards, etc.]. As an example, the reader is referred to Green (2002), a study estimating bed unavailability in obstetrics and ICUs units; de Bruin et al. (2007), which included ED, ICU and cardiac care wards; Cochran and Roche (2008), an analysis of hospital bed capacity at four care levels (ICU, telemetry, medical/surgical and obstetrics); and de Bruin et al. (2010), a sizing analysis of 24 wards in a medical center, including general and critical care. Nevertheless, there are few papers that focus on ICUs: Shmueli et al. (2003) used queuing theory to find optimal admission policies; McManus et al. (2004) analyzed bed availability, utilization rates and rejection rates using queuing theory; and Griffiths et al. (2013) proposed a queuing model to improve bed management distinguishing between emergency and elective surgery patients.

Queuing theory is a powerful analytical tool for building simple models with relatively few little data, while including randomness. Simulation is another common approach in queuing modelling, which enables a more detailed representation of the complexity in health systems. Reviews and discussion papers dealing with the application of simulation modelling in health care can be found in Brailsford et al. (2009), Günal and Pidd (2010), Eldabi et al. (2007) and Katsaliaki and Mustafee (2011). Many studies use simulation to analyze hospital capacity and bed allocation, but, again, only a few deal specifically with ICUs. Worth noting are Kim et al. (1999, 2000), in which ICU admission and discharge processes are analyzed through simulation and several bed allocation rules are evaluated; Litvak et al. (2008), Ridge et al. (1998), Costa et al. (2003) and Zhu et al. (2012), in which the ICU capacity problem is studied; Masterson et al. (2004) which presents a case study of an ICU at a military medical centre analyzing staffing, sizing and operational policies; and Kolker (2009), in which an ICU simulation model is used to establish a quantitative link between the daily load levelling of elective surgeries and ICU diversion.

Queue system performance (particularly in an ICU) is greatly influenced by the client (patient) arrival pattern and the service time (length of stay in the ICU). The ICU arrival pattern usually consists of a mix of emergency and elective patients. Emergency patient arrivals are typically modeled as Poisson processes (see, for example, Griffiths et al. (2005), de Bruin et al. (2010)), including patient arrivals from different sources (emergency surgery, ward, emergency room, another hospital, etc.). Elective arrivals, however, are scheduled and therefore very unlikely to follow a Poisson process. While some authors use other arrival distributions for scheduled patients: empirical distributions (Mallor and Azcárate (2014)), non-stationary Poisson processes (Griffiths et al. (2005)), and deterministic distributions (Zhu et al. (2012)), other consider it reasonable to assume Poisson arrivals [see for example Litvak et al. (2008) or Kim et al. (2000)]. Inter-arrival exponential distribution is taken in Green (2002), Shmueli et al. (2003), McManus et al. (2004), Griffiths et al. (2006) and Troy and Rosenberg (2009).

Patient length of stay (LoS) has also been extensively studied (Zimmerman et al. 2006). Some authors consider exponential distributions to approximate patient LoS in ICU analytical approaches. Examples can be found in the model proposed by McManus et al. (2004) and that presented by Shmueli et al. (2003). Kim et al. (1999) assume exponentially distributed service times for all the patient groups included in their simulation model. Nevertheless, several studies have proved that LoS distributions are usually heavily skewed to the right (see for example, Rauner et al. (2003) and Vasilakis and Marshall (2005)). These empirical findings support the choice of weighted-tail distributions for LoS modeling purposes: Weibull distributions are used in Ridge et al. (1998); Lognormal distributions in Costa et al. (2003) and in Masterson et al. (2004); Pearson VI in Griffiths et al. (2005); phase-type distributions in Griffiths et al. (2006) and Bowers (2013). Mallor and Azcárate (2014) proposed non-normal regression models including variables with the power to explain some of the LoS variability, such as the Apache index. An overview of LoS and patient flow modeling techniques can be found in Marshall et al. (2005).

The stochastic nature of both patient arrivals and LoS periodically lead to inevitable bed shortages. If all beds are occupied, eligible patients are usually referred to another ward or hospital and scheduled surgeries can be cancelled. Bed shortages may also have consequences for ICU patients, who may be eligible for early discharge when the number of occupied beds is so high as to compromise the medical care of new incoming patients. Therefore, both patient admissions and discharges are triaged (Shmueli et al. (2003), Sinuff et al. (2004), Capuzzo et al. (2010) and Anderson et al. (2011), Chan et al. (2012, 2014).

Ridge et al. (1998) pointed out that the “early discharge” of less critical patients to other wards is a solution commonly adopted to cope with ICU bed shortages. Costa et al. (2003) observed the dynamical changes in ICU management when units become full and physicians attempt to limit admissions or to discharge patients who are closer to recovery. Although these studies suggest early discharge as a bed management tool, they do not include it in their mathematical models. The impact of workload on service time in health care has been considered by Kc and Terwiesch (2009), who show that hospital resources are sensitive to load levels and that service workers can adapt to system needs by increasing the service rate. Mallor and Azcárate (2014) demonstrated in a real setting that patient LoS is not independent of the ICU workload and may be influenced by the ICU bed occupancy level. As a consequence, they signalled the need to include these discharge policies to obtain a valid simulation model.

The bed capacity planning problem has drawn much attention in ICU management studies, where various economic and quality-of-service (QoS) criteria have been considered. They include minimization of total operating costs, full demand coverage, reasonable bed utilization rates, a lower rejection rate, minimization of the number of surgery cancelations, maximization of the expected incremental number of lives saved. We consider the following two QoS criteria. The first is of a social nature: i.e., the percentage of population that can benefit from the ICU care when needed. The second is of an individual nature: i.e., the degree of recovery reached by a patient on discharge from the ICU. Thus, high quality of service in an ICU means both a low percentage of rejected patients and sufficient LoS.

This paper deals with the problem of obtaining efficient bed-management policies to control the ICU bed occupancy level. Queuing theory addresses the problem of resource allocation under uncertainty (Gross and Harris 2008) to provide queue designs and control policies that optimize some measure of interest. Here, we consider a different queuing control problem in which neither the arrival rates nor the number of servers can be modified. The bed occupancy control problem is addressed by adjusting the service time rates to make them dependent of the state of the system. Best values for these service times must then be determined to achieve the two QoS targets already mentioned: i.e., minimal patient rejection (due to full ICU occupancy) and minimal shortening of the LoS. The first objective has a clear mathematical formulation: minimize the probability of rejecting a patient. The second objective admits several formulations, which will be discussed later. Four different expressions, all leading to nonlinear optimization problems, are proposed. The analytical solutions to these problems-which define the desired management policies- are obtained and compared in terms of the two QoS components.

The paper is organized as follows. Section 2 presents a Markovian analysis of the ICU and provides some theoretical results for the ICU queuing model. We focus on the mathematical modelling of the ICU control problem, considering the two QoS components already mentioned. We propose four different nonlinear multi-objective problems for the Markovian case and give the efficient management policies based on the analytical solutions to the different control problems. Section 3 presents the extension of the control problems and the management policies to the general G/G/c/c model and proposes a hazard function method to simulate general LoS distributions with service time dependent on the number of occupied beds. The optimal management policies are obtained by combining optimization with simulation. Finally, Sect. 4 contains the conclusions and final remarks regarding our work in progress and some suggestions for future research.

2 Markovian analysis for an ICU

2.1 Queuing modelling and analysis

As mentioned in the introduction, an ICU can be mathematically modelled as a queuing system, where the clients are the patients, the servers are the beds and there is no waiting room. Then a general model that fits this system is a G/G/c/c queue model with several types of customers, each characterised by an arrival pattern and a service time. The complexity of this model generally demands the use of simulation to obtain a full and detailed probabilistic analysis. We first study a simplified version of the ICU queuing model by adopting Markovian assumptions on arrivals and service times. These assumptions could be considered not totally unrealistic in an ICU serving only emergency patients. Specifically, we assume a M/M/c/c queue model where arrival rate is a constant λ and the service rates $\varvec{\mu}= (\mu_{1} , \ldots , \mu_{i} , \ldots , \mu_{c} )$ can be dependent on the state of the system i, that is, dependent on the number i of occupied beds.

Let X(t) be the variable denoting the number of occupied beds at time t and $\varvec{P} = (p_{0} , p_{1} , \ldots , p_{c} )$ the stationary probability distribution of process $\left\{ {X(t)} \right\}$ when the service is provided according to service rates $\varvec{\mu}= (\mu_{1} , \ldots , \mu_{i} , \ldots , \mu_{c} )$.

Proposition

Given a constant arrival ratio λ and a probability distribution $\varvec{P} > 0$, there is a unique set of service rates $\varvec{\mu}= (\mu_{1} , \ldots , \mu_{c} )$ such that the stationary probability distribution of process $\left\{ {X(t)} \right\}$ is $\varvec{P}$.

Proof

Straightforward from steady state equilibrium equations:

$$\mu_{i} = \lambda \frac{{p_{i - 1} }}{{ip_{i} }}\quad \forall i = 1, \ldots , c \quad {\text{and}}\quad \mathop \sum \limits_{i = 0}^{c} p_{i} = 1$$

□

This proposition shows that there exist service-rate values leading to any probability distribution of the number of occupied beds.

Let us consider a queuing model with input and service rates λ and μ, respectively, and denote the steady state probabilities by $p_{i}^{*} \; i = 0, \ldots , c$. Suppose that a new target distribution for the state probabilities $p_{i} ,\,i = 0, \ldots ,c$, is desired for the queueing model and that this new distribution has to be achieved by determining new, state-dependent service rates.

Then, by the previous proposition, the new service rates $\mu_{i} \,i = 1, \ldots ,c$ verify the expression:

$$\frac{{\mu_{i} }}{\mu } = \frac{{{{p_{i - 1} } \mathord{\left/ {\vphantom {{p_{i - 1} } {p_{i} }}} \right. \kern-0pt} {p_{i} }}}}{{{{p_{i - 1}^{*} } \mathord{\left/ {\vphantom {{p_{i - 1}^{*} } {p_{i}^{*} }}} \right. \kern-0pt} {p_{i}^{*} }}}} = \frac{{{{p_{i - 1} } \mathord{\left/ {\vphantom {{p_{i - 1} } {p_{i - 1}^{*} }}} \right. \kern-0pt} {p_{i - 1}^{*} }}}}{{{{p_{i} } \mathord{\left/ {\vphantom {{p_{i} } {p_{i}^{*} }}} \right. \kern-0pt} {p_{i}^{*} }}}} = \frac{{{\upvarphi }_{i - 1} }}{{{\upvarphi }_{i} }}$$

(1)

where $\upvarphi_{i}$ denotes the relative change with respect to the initial probabilities:

$$\upvarphi_{i} = {{p_{i} } \mathord{\left/ {\vphantom {{p_{i} } {p_{i }^{*} }}} \right. \kern-0pt} {p_{i }^{*} }}$$

This result relates the necessary changes in the service rates with the magnitude of change in the probabilities and shows that this change is a continuous function:

$${{{{\forall \varepsilon > 0, \;\exists \delta > 0} \mathord{\left/ {\vphantom {{\forall \varepsilon > 0, \;\exists \delta > 0} { \left| {\mu_{i} - \mu } \right| < \varepsilon , \;\forall p_{i }^{*} }}} \right. \kern-0pt} { \left| {\mu_{i} - \mu } \right| < \varepsilon , \;\forall p_{i }^{*} }} \,} \mathord{\left/ {\vphantom {{{{\forall \varepsilon > 0, \;\exists \delta > 0} \mathord{\left/ {\vphantom {{\forall \varepsilon > 0, \;\exists \delta > 0} { \left| {\mu_{i} - \mu } \right| < \varepsilon , \;\forall p_{i }^{*} }}} \right. \kern-0pt} { \left| {\mu_{i} - \mu } \right| < \varepsilon , \;\forall p_{i }^{*} }} \,} { \left| {p_{i }^{*} - p_{i} } \right| < \delta }}} \right. \kern-0pt} { \left| {p_{i }^{*} - p_{i} } \right| < \delta }}$$

Let us now consider the LoS of a patient in an ICU when the service rates are dependent of the number of occupied beds. The transitions of the patient among the different states of the queuing system can be modeled as a continuous-time Markov process with an absorbing state representing the patient’s discharge from the health facility.

Thus, a patient remains at state i during an exponentially distributed time with rate $(\lambda + i\mu_{i} )$ and then jumps to

• state i + 1 with probability $\lambda /(\lambda + i\mu_{i} )$

• state i − 1 with probability ${{(i - 1)\mu_{i} } \mathord{\left/ {\vphantom {{(i - 1)\mu_{i} } {(\lambda + i\mu_{i} )}}} \right. \kern-0pt} {(\lambda + i\mu_{i} )}}$

• the absorbing state with probability ${{\mu_{i} } \mathord{\left/ {\vphantom {{\mu_{i} } {(\lambda + i\mu_{i} )}}} \right. \kern-0pt} {(\lambda + i\mu_{i} )}}$

The LoS of a patient can be seen as the absorption time of that continuous-time Markov process in which the initial distribution for the states is the bed occupancy stationary distribution conditioned to a state smaller than c. We denote this initial distribution by vector $\varvec{\alpha}$. Therefore, the LoS, denoted by variable Z, follows a Phase-type distribution characterized by the following transition rate matrix:

This matrix can be expressed as $\left( {\begin{array}{*{20}c} \varvec{T} & {{\text{T}}_{0} } \\ \varvec{0} & 0 \\ \end{array} } \right)$

Then it is well known that the distribution function of the LoS, Z, verifies the following expression:

$$P\left\{ {Z \le x} \right\} = F_{Z} \left( x \right) = 1 -\varvec{\alpha}exp\left\{ {\varvec{T}x} \right\}\varvec{e}$$

where e is the vector with all components equal to 1.

The conditional stationary distribution $\varvec{\alpha}$ is expressed in terms of the steady state probabilities $p_{i}$ as

$$\varvec{\alpha}= \left( {\varvec{ }{\raise0.7ex\hbox{${p_{0} }$} \!\mathord{\left/ {\vphantom {{p_{0} } {1 - p_{c} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${1 - p_{c} }$}},{\raise0.7ex\hbox{${p_{1} }$} \!\mathord{\left/ {\vphantom {{p_{1} } {1 - p_{c} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${1 - p_{c} }$}}, \ldots , {\raise0.7ex\hbox{${p_{c - 1} }$} \!\mathord{\left/ {\vphantom {{p_{c - 1} } {1 - p_{c} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${1 - p_{c} }$}}} \right)$$

which leads to the following nonlinear expression in terms of $\mu_{i} i = 1, \ldots ,c$:

$$\varvec{\alpha}= \left( { {\raise0.7ex\hbox{${\gamma_{1} }$} \!\mathord{\left/ {\vphantom {{\gamma_{1} } \theta }}\right.\kern-0pt} \!\lower0.7ex\hbox{$\theta $}},{\raise0.7ex\hbox{${\gamma_{2} }$} \!\mathord{\left/ {\vphantom {{\gamma_{2} } {\theta }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\theta }$}}, \ldots , {\raise0.7ex\hbox{${\gamma_{\text{c}} }$} \!\mathord{\left/ {\vphantom {{\gamma_{\text{c}} } {\theta }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\theta }$}}} \right)$$

where:

$$\theta = \lambda^{c - 1} + \mathop \sum \limits_{i = 1}^{c - 1} \lambda^{c - i - 1} \frac{(c - 1)!}{{\left( {c - i - 1} \right)!}} \mathop \prod \limits_{j = c - i}^{c - 1} \mu_{j}$$

and

$$\gamma_{i} = \lambda^{i - 1} \frac{(c - 1)!}{{\left( {i - 1} \right)!}} \mathop \prod \limits_{j = i}^{c - 1} \mu_{j} \quad i = 1, \ldots ,c$$

The first two moments of the random variable Z representing the LoS are given by:

$${\text{E}}\left[ {\text{Z}} \right] = - (\varvec{\alpha T}^{ - 1} \varvec{ e})$$

$${\text{E}}\left[ {{\text{Z}}^{2} } \right] = 2(\varvec{\alpha T}^{ - 2} \varvec{ e})$$

Let be $\varvec{\alpha}_{i}$ a c-dimensional vector with 1 in position i + 1 and the rest equal to 0. The substitution of vector $\varvec{\alpha}$ by $\varvec{\alpha}_{i}$ in the above expressions provides the distribution function of variable $\varvec{Z}_{(i)}$ representing the LoS conditioned on the patient having entered the system at state i, $F_{{Z_{(i)} }} \left( x \right) = 1 -\varvec{\alpha}_{\varvec{i}} exp\left\{ {\varvec{T}x} \right\}\varvec{e}$, and the first two moments:

$${\text{E}}\left[ {\varvec{Z}_{(i)} } \right] = - (\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 1} \varvec{ e})$$

$${\text{E}}\left[ {Z_{(i)}^{2} } \right] = 2(\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 2} \varvec{ e})$$

These expressions are used in next section in the mathematical definition of the queue control problems.

2.2 Mathematical modeling of control problems

Queuing theory has studied the problem of resource allocation under uncertainty (Gross and Harris 2008) to provide queue designs and control policies that optimize some measure of interest, such as a customer’s expected waiting time. One example of this type of control problem is the determination of a policy for switching workers in a facility with front room and back room operations in order to cope with changing customer demand (Terekhov and Beck 2008, and references therein). Another is how to control the queue by developing optimal admission policies taking customer behavior characteristics into account, which has led to the idea of controlling arrivals by pricing (see Stidham 2002).

We consider a type of control problem in which the objective is to maximize QoS (by reducing as far as possible the probability of patient rejection and to minimize the shortening of patient’s LoS). Nevertheless, two ICU characteristics need to be taken into account: fixed resources and unadjustable patient arrival rates. Thus, in our queuing control problem, neither the arrival rate nor the amount of resources can be modified. Therefore, bed occupancy is controlled by modifying service time rates to make them state-dependent.

In this section, we study this control problem in a M/M/c/c queue model with constant arrival rate λ and “normal” service rate μ which can be adjusted to individual service rate values $\mu_{i}$, when i beds are occupied, $i = 1, \ldots ,c$. Then, the aim of the control problem is to determine new values for service rates $\mu_{i}, i = 1, \ldots ,c$, that minimize the probability of rejecting a new patient with minimum adjustment of the LoS of an admitted patient.

The first objective has a clear mathematical formulation: $Minimize \;p_{c}$. By the PASTA property (Poisson Arrivals See Time Averages), $p_{c}$ is the probability of a patient finding a full ICU on arrival. The second objective admits different formulations. We propose two approaches, one based on the variation in services rates and another on the variation in the expected LoS. Expressions (2) and (3) are two alternative mathematical formulations for the “variation in services rates” objective functions. Expressions (4) and (5) are proposed for “variation in the expected LoS” objectives.

The first approach considers the magnitude by which the original service rate μ is adjusted in order to obtain the new service rates $\mu_{i} \; i = 1, \ldots ,c$:

$$min\,max_{i} \left| {\mu_{i} - \mu } \right|$$

(2)

or

$$min\mathop \sum \limits_{i = 1}^{c} \left| {\mu_{i} - \mu } \right|$$

(3)

Expression (4) minimizes the difference between the expected LoS of a patient in the ICU “with no control policy” (constant service rate μ) and the expected LoS “with control policy” (service rate $\mu_{i} \;i = 1, \ldots ,c$ depending on the bed occupancy level i):

$$\hbox{min} \left| { - \left( {\varvec{\alpha T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right|$$

(4)

Expression (5) modifies (4) by considering the patient’s expected LoS conditioned by the state i of the ICU at arrival time.

$$min\,max_{i} \left| { - \left( {\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right|$$

(5)

Each of these formulations for the second objective leads to a different multi-objective optimization problem:

$$\begin{gathered} min\,P_{c} \hfill \\ {\text{min }}max_{i} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\,to\,\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} \hfill \\ \end{gathered}$$

(MOP1)

$$\begin{gathered} min\,P_{c} \hfill \\ \hbox{min} \mathop \sum \limits_{i = 1}^{c} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\; to\; \mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} \hfill \\ \end{gathered}$$

(MOP2)

$$\begin{gathered} min\,P_{c} \hfill \\ \hbox{min} \left| { - \left( {\varvec{\alpha T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right| \hfill \\ subject\; to\; \mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} \hfill \\ \end{gathered}$$

(MOP3)

$$\begin{gathered} min\,P_{c} \hfill \\ \hbox{min}\,max_{i} \left| { - \left( {\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right| \hfill \\ subject\; to\; \mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} \hfill \\ \end{gathered}$$

(MOP4)

Observe that monotonicity constraints on the values of $\mu_{i}$ reflect the fact that service times become shorter as the ICU gets busier.

We estimate the Pareto frontier by using the ε-constraint method and considering different bounds for the probability of rejected patients ($\varepsilon_{j}$-values):

$$\begin{gathered} {\text{min }}max_{i} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\; to\;\left\{ {\begin{array}{*{20}l} {P_{c} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(P1)

$$\begin{gathered} \hbox{min} \mathop \sum \limits_{i = 1}^{c} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\,to\,\left\{ {\begin{array}{*{20}l} {P_{c} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(P2)

$$\begin{gathered} { \hbox{min} }\left| { - \left( {\varvec{\alpha T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right| \hfill \\ subject\;to\; \left\{ {\begin{array}{*{20}l} {P_{c} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(P3)

$$\begin{gathered} \hbox{min} \,max_{i} \left| { - \left( {\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 1} \varvec{ e}} \right) - \frac{1}{\mu }} \right| \hfill \\ subject\; to\; \left\{ {\begin{array}{*{20}l} {P_{c} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(P4)

All these models are nonlinear optimization problems with the following structure:

All optimization problems have c decision variables ($\mu_{1} ,\mu_{2} , \ldots , \mu_{c}$). The nonlinearity of the problems comes from the absolute value functions and from the term $P_{c}$, which is given by the following nonlinear expression:

$$P_{c} = \frac{{\lambda^{c} }}{{\lambda^{c} + \mathop \sum \nolimits_{i = 1}^{c} ( \lambda^{c - i} i!\left( {\begin{array}{*{20}c} c \\ i \\ \end{array} } \right) \mathop \prod \nolimits_{j = c - i + 1}^{c} \mu_{j} )}}$$

(6)

Optimization problem (P3) also includes the following nonlinear function:

$$\left( {\varvec{\alpha T}^{ - 1} \varvec{ e}} \right) = \frac{{\beta_{1} + \mathop \sum \nolimits_{i = 2}^{c} \beta_{i} \mathop \prod \nolimits_{j = c - i + 2}^{c} \mu_{j} }}{{\mathop \sum \nolimits_{i = 1}^{c} \beta_{i} \mathop \prod \nolimits_{j = c - i + 1}^{c} \mu_{j} }}$$

(7)

where $\beta_{i}$ are constant values depending on $\lambda$ and c.

Expression $\left( {\varvec{\alpha}_{\varvec{i}} \varvec{ T}^{ - 1} \varvec{ e}} \right)\varvec{ }$ in optimization problem (P4) is a complex, nonlinear function, which, for i = c, includes a monomial for each of the proper subsets of $\mu_{1} ,\mu_{2} , \ldots , \mu_{c}$, that is, $2^{c} - 2$ terms.

Furthermore, (P1) and (P4) have min max objective functions.

Observe that some of the nonlinear terms in the above problems admit reformulation. For example, minmax objective functions admit linear reformulation by considering an additional variable ($\delta$), as illustrated in the following expression for the problem (P1) case:

$$\begin{array}{*{20}l} {\hbox{min}\,\delta } \hfill \\ {subject\; to\; \left\{ {\begin{array}{*{20}l} {\left| {\mu_{i} - \mu } \right| \le \delta \quad i = 1, \ldots ,c} \hfill \\ {P_{c} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \ldots \le \mu_{c} } \hfill \\ \end{array} } \right.} \hfill \\ \end{array}$$

Let $dev_{i} , i = 1, \ldots , c$ be the deviational variables associated with the deviation from 1 of ratios $\mu_{i} /\mu , \;i = 1, \ldots ,c$. Optimization problems (P1) and (P2) can also be reformulated [(P1b) and (P2b)] by considering both the steady state equilibrium equations and expression (1).

$$\hbox{min}\,max_{j}\,dev_{j}$$

(P1b)

or

$$\hbox{min} \mathop \sum \limits_{j = 1}^{c} dev_{j}$$

(P2b)

$$subject \;to\;\left\{ {\begin{array}{*{20}l} {\varphi_{c} P_{c}^{*} \le \varepsilon_{j} } \hfill \\ {\frac{{\varphi_{i - 1} }}{{\varphi_{i} }} - dev_{i} = 1,\quad i = 1, \ldots , c} \hfill \\ {\mathop \sum \limits_{i = 0}^{c} \varphi_{i} P_{i}^{*} = 1} \hfill \\ {\frac{{\varphi_{i - 1} \varphi_{i + 1} }}{{\varphi_{i}^{2} }} \le 1, \quad \forall i = 1, \ldots , c } \hfill \\ \end{array} } \right.$$

This nonlinear formulation has 2c + 1 variables ($\varphi_{0} ,.., \varphi_{c} ,dev_{1} , \ldots , dev_{c}$) and avoids the calculation of the term $P_{c}$ in expression (6).

2.3 Efficient management policies for the control problem

Management policies are obtained from the solution of the mathematical problems formulated in the previous section. Different optimization problems lead to different solutions and then to different management policies. Solutions differ not only in service rate values but also in structure, with management philosophy implications. In some cases, the only dramatical increase in the service rate occurs at full occupancy, while, in the other cases, the service rates are moderately and evenly increased in all states. We label these two types of policies aggressive and equitable, respectively. Another, intermediate structure is also found: the higher the occupancy level, the higher the service rate. We label this last policy as cautious.

To illustrate these results we study a small ICU with five beds and then a larger one with 20 beds, based on a real ICU analyzed in a previous study by the authors.

The 5-bed ICU is represented by the M/M/5/5 queuing model. Without loss of generality, in our study we consider the time unit as the average LoS, so that μ = 1. Then, the different studied scenarios are defined by varying the values of λ to achieve certain desired occupancy ratios. In this way values of μ = 1 and λ = 4, 3 and 2 lead to occupancy rates of 80, 60 and 40 %, respectively. The four optimization problems are solved in each scenario. Optimization problems (P3) and (P4) always provide aggressive policies, (P1) leads to equitable policies and (P2) leads to cautious policies. Results are presented in Tables 1, 2 and 3.

Table 1 Optimal management policies for a M/M/5/5, with λ = 4 and $\mu$ = 1 (server utilization = 80 %)

Table 2 Optimal management policies for a M/M/5/5, with λ = 3 and $\mu$ = 1 (server utilization = 60 %)

Table 3 Optimal management policies for a M/M/5/5, with $\lambda$ = 2 and μ = 1 (server utilization = 40 %)

Table 1 shows the results for λ = 4 (server utilization = 80 %) and $P_{5}^{*} = 0.199067$. Recall that this means that 19.9 % of patients are rejected due to full ICU occupancy, in the absence of any control by physicians. To estimate the Pareto frontier by using the ε-constraint method, we consider the following $\varepsilon_{j}$-values, $\varepsilon_{j}$ = 19, 16, 13, 10, 7, 4 and 1 %. Table 2 shows the results for λ = 3 (server utilization = 60 %) and $P_{c}^{*} = 0.110054$, considering the following $\varepsilon_{j}$-values, $\varepsilon_{j}$ = 10, 8, 6, 4, 2 and 1 %. Table 3 shows the results for λ = 2 (server utilization = 40 %) and $P_{c}^{*} = 0.036697$, considering the following $\varepsilon_{j}$-values, $\varepsilon_{j}$ = 3, 2.5, 2, 1.5, 1 and 0.5 %.

In these results, we see the three policy structures previously defined:

• Aggressive policy: $\mu_{1} = \mu_{2} = \cdots = \mu_{c - 1} = 1$ and $\mu_{c} \gg 1$ in (P3) [and also (P4)] results.

• Equitable policy: $\mu_{1} = \mu_{2} = \cdots = \mu_{c} > 1$ in (P1) results.

• Cautious policy: $1 \le \mu_{1} \le \mu_{2} \le \cdots \le \mu_{c}$ in (P2) results.

The same pattern appears in the results for the simplified version of a real ICU (the Hospital of Navarre, Spain) with 20 beds (M/M/20/20 queuing model).

In this case, we simplify the problem by distinguishing three levels for the bed occupancy state:

• low occupancy (<50 %): 1–10 occupied beds,

• moderate occupancy (50–75 %): 11–14 occupied beds,

• high occupancy (≥75 %): 15–20 occupied beds.

We assume:

$$\mu_{1} = \mu_{2} = \cdots = \mu_{10} \le \mu_{11} = \mu_{12} = \cdots = \mu_{14} \le \mu_{15} \le \mu_{16} \le \mu_{17} \le \mu_{18} \le \mu_{19} \le \mu_{20}$$

This assumption reduces the number of decision variables, thereby simplifying both the computational analysis and the presentation of results.

Table 4 includes the results for λ = 16 and $\mu$ = 1 (server utilization = 80 %) and $P_{c}^{*} = 0.064411$. We consider the following $\varepsilon_{j}$-values, $\varepsilon_{j}$ = 6, 5, 4, 3, 2, 1 %.

Table 4 Optimal management policies for the Markovian version of the ICU of the Hospital of Navarre, with 20 beds

The question that clearly arises is which of the three types of ICU management policy is the best. Each is optimal in terms of one specific aspect of LoS shortening but none is optimal in all the LoS shortening objectives. Determining which is the best or the most appropriate management policy is therefore no easy task. The decision must be based on the performance assessment of each policy type in all the different LoS shortening-related measures and on the judgment of physicians.

Comparison of the three management strategies shows that cautious policies are the best for their associated objective ($min\sum\nolimits_{i = 1}^{c} {\left| {\mu_{i} - \mu } \right|}$) and the second best for the other objectives. Aggressive policies are the best for the objectives associated with maximizing average LoS, the second best for minimizing the maximum deviation from the original service rate and the worst for the sum of service rate deviations. Equitable policies are the best for minimizing maximum deviation from the original service rate, the second best for the sum of service rate deviations, and the worst for the objectives associated with maximizing average LoS. See Fig. 1. From this analysis, cautious policies appear to display the best behavior overall. Furthermore, ICU physicians agree that cautious policies are the best choice in bed shortage situations. In fact, the representation of the bed-occupancy distribution under the different policies (Fig. 2) shows that the one corresponding to the cautious policy is the closest to histograms built from historical occupancy data collected in real ICUs [see, for example, Costa et al. (2003) and Mallor and Azcárate (2014)]. Common characteristics in both the cautious distribution and real data distributions are a sharp decline on the right side of the histogram and a skewed histogram with a peak around a set of desirable bed occupancy values (leading to high utilization of resources without compromising the entry of new patients).

Fig. 1

Pareto frontiers for the optimal management policies

Fig. 2

Bed occupancy distribution plots for the different optimal management policies. Scenario with 1 % probability of rejecting a patient

3 Control problems and management policies in the general case G/G/c/c

Section 2 modeled the control problem of a health care service through a set of optimization problems which were solved by means of classical mathematical programming. This setting could be applied to the Markovian case. When other, non-exponential distributions are considered, the analytical expressions for the second objective in problems (P3) and (P4) do not hold and even the decision variables μ_i need to be reinterpreted. This section addresses these questions, first by assessing the combination of optimization with simulation as an appropriate optimization problem-solving tool and then by providing a way to simulate general LoS distributions allowing the inclusion of state-dependent service rates. We then present the management policies obtained from this new optimization and simulation framework when different Weibull-type LoS distributions are considered and compare them with those obtained in the Markovian case.

3.1 Finding optimal policies by combining optimization with simulation

Optimization problems (P1) to (P4) include the expected LoS, E[Z], the expected conditional LoS, E[Z _(i) ], and the probability of rejecting a patient, P _rej , which have an explicit expression in terms of the decision variables in the Markovian case but not in the general case. The formulation of the optimization problems for the general case is:

$$\begin{gathered} {\text{min }}\,max_{i} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\, to\,\left\{ {\begin{array}{*{20}l} {P_{rej} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(GP1)

$$\begin{gathered} \hbox{min} \mathop \sum \limits_{i = 1}^{c} \left| {\mu_{i} - \mu } \right| \hfill \\ subject\, to\,\left\{ {\begin{array}{*{20}l} {P_{rej} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(GP2)

$$\begin{gathered} \hbox{min} \left| {{\text{E}}[{\text{Z}}] - \frac{1}{\mu }} \right| \hfill \\ subject \,to\,\left\{ {\begin{array}{*{20}l} {P_{rej} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(GP3)

$$\begin{gathered} \hbox{min}\,max_{i} \left| {{\text{E}}[{\text{Z}}_{(i)} ] - \frac{1}{\mu }} \right| \hfill \\ subject\;to \;\left\{ {\begin{array}{*{20}l} {P_{rej} \le \varepsilon_{j} } \hfill \\ {\mu_{1} \le \mu_{2} \le \cdots \le \mu_{c} } \hfill \\ \end{array} } \right. \hfill \\ \end{gathered}$$

(GP4)

Now objective functions in (GP1) and (GP2) and the constraint on the probability of rejecting a patient $P_{rej} \le \epsilon_{j}$ have to be assessed by simulation. To solve this set of problems, we combine simulation with an optimization procedure, which determines a solution, that is, a value for the decision variables $\mu_{i}$ of the problem which define the configuration of the simulated system. The output of the simulation is used to evaluate the stochastic elements and assess the quality of the current solution. The optimization procedure, with this information and its search method, provides the next solution to be evaluated by simulation. This iterative process goes on until the stopping conditions of the optimization method are met. The simulation-based optimization (SBO) methodology just described has been widely applied to solve stochastic optimization problems in different contexts, including healthcare. For example, in de Angelis et al. (2003) a simulation model is combined with nonlinear programming and neuronal networks to determine the optimal configuration of a transfusion center; in Azcárate et al. (2008) a multicriteria optimization simulation model is proposed to solve a hospital sizing problem; in Ahmed and Alkhamis (2009) one is used to obtain the optimal staff distribution for a hospital emergency unit; in Brailsford et al. (2007) a discrete-event simulation model is embedded in an ant colony optimization model for the optimal choice of screening policies for diabetic retinopathy; in Lin et al. (2013) simulation is combined with a genetic algorithm and data envelopment analysis to determine optimal resource levels in surgical services. The reader is referred to Fu et al. (2005) for a descriptive review of the main approaches for SBO.

The SBO methodology to solve the above control problems is validated by solving the Markovian case and comparing the results with the analytical ones. The SOB technique was implemented in ARENA simulation software and OptQuest optimization software. Figure 3 plots Pareto frontiers for optimization problems obtained by nonlinear programming techniques (black lines) and simulation and optimization techniques (red lines).

Fig. 3

Pareto frontiers for optimization problems obtained by nonlinear programming techniques (black lines) and simulation and optimization techniques (red lines)

Figure 4 plots the relative error between analytical and simulated optimal solutions, expressed as

$$\frac{{\left| {\mu_{{analytical_{i} }} - \mu_{{simul_{i} }} } \right|}}{{\mu_{{analytical_{i} }} }}$$

Fig. 4

Relative error between analytical and simulated optimal solutions

It can be observed that, in general, simulation provides a good estimation of the analytical solutions. The relative error in almost all cases is less than 3 %, only a few cases for (GP2) show values between 4 and 6 %. Comparing the optimal objective functions, the differences between analytical and simulated results are less than 0.01. Thus, we can conclude that the SBO methodology is able to solve the proposed control problems and obtain the management policies.

3.2 Simulation from general distributions with varying service rates

The application of the SBO methodology outlined in previous section requires the simulation of queuing systems with varying service rates. In the Markovian case, the procedure is straightforward due to the lack of memory of the exponential distribution. When the system enters a new state i, which has to operate with a service rate of $\mu_{i}$, the remaining LoS of each patient is updated by sampling a value from an exponentially-distributed random variable $T_{{\mu_{i} }}$ with mean $1/\mu_{i}$. Assuming that the LoS is initially distributed as an exponential random variable $T_{\mu }$ with mean $1/\mu$, the variation to another service rate $\mu_{i}$ can be interpreted as a change in the time scale:

$$P\left\{ {T_{{\mu_{i} }} \le t} \right\} = 1 - e^{{ - \mu_{i} t}} = 1 - e^{{ - \frac{{\mu_{i} }}{\mu } \mu t}} = P\left\{ {T_{\mu } \le \frac{{\mu_{i} }}{\mu }t} \right\} = P\left\{ {\frac{\mu }{{\mu_{i} }} T_{\mu } \le t} \right\}$$

Then, $T_{{\mu_{i} }} = \frac{\mu }{{\mu_{i} }} T_{\mu }$ and $F_{{T_{{\mu_{i} }} }} (t) = F_{{T_{\mu } }} \left( {\frac{{\mu_{i} }}{\mu } t} \right)$.

When the system enters state i with an associated service rate $\mu_{i}$ the remaining LoS of that patient is adjusted by a scale factor of $\mu /\mu_{i}$.

This relationship is extended to the hazard function:

$$h_{{T_{{\mu_{i} }} }} \left( t \right) = \frac{{F_{{T_{{\mu_{i} }} }}^{\prime } (t)}}{{1 - F_{{T_{{\mu_{i} }} }} \left( t \right)}} = \frac{{F_{{T_{\mu } }}^{\prime } (\frac{{\mu_{i} }}{\mu } t)}}{{1 - F_{{T_{\mu } }} \left( {\frac{{\mu_{i} }}{\mu } t} \right)}} = \frac{{\mu_{i} }}{\mu }h_{{T_{\mu } }} \left( {\frac{{\mu_{i} }}{\mu }t} \right)$$

(8)

The hazard function at time t describes the probabilistic behaviour of a random variable at time t conditioned to a value greater than t:

$$h_{{T_{{\mu_{i} }} }} \left( t \right) = \mathop {\lim }\limits_{{\Delta {\text{t}} \to 0}} \frac{{{\text{P}}\{ {\text{t}} < {\text{T}}_{{\mu_{i} }} \le {\text{t}} +\Delta {\text{t}}/ {\text{T}}_{{\mu_{i} }} > t\} }}{{\Delta {\text{t}}}}$$

Then, the probability that a patient’s LoS falls within the time interval $(t, t +\Delta \varvec{t}]$, given that at time t the patient is still in the health system, can be approximated by $\frac{{h_{{T_{{\mu_{i} }} }} \left( {t +\Delta {\text{t}}} \right) + h_{{T_{{\mu_{i} }} }} \left( t \right)}}{2} \Delta t$.

This expression is obtained from the two first terms of the Taylor series expansion [(1) in (9)] and numerical integration by trapezoidal method [(2) in (9)].

(9)

This interpretation of the hazard function, together with expression (8), is used to simulate the LoS patients in the health system when both the general LoS distributions and varying service rate are considered.

As usual, the discrete event simulation model considers both patient arrival and end of service as state-changing events. However, the simulation is now driven by the patient LoS hazard functions as follows:

Step 0

Choose a small value for $\Delta t$.

Step 1

Simulate a time for the next patient arrival.

Step 2

If less than $\Delta t$ time units left to a new patient arrival, then the system is updated, the clock is advanced to the arrival time of the new patient and step 1 is repeated. Let $t_{e}$ be the patient´s accelerated time in system. At the entry time of a patient, $t_{e} = 0$.

Step 3

If the system is in state i at time t (bed occupancy level is i) the hazard function $h_{{T_{{\mu_{i} }} }} \left( {t_{e} } \right)$ of the LoS of each patient is calculated, according to expression (8).

Step 4

For each patient the probability of leaving the health system in the next time interval of length $\Delta t$ is calculated according to expression (9) and is used to decide whether the patient leaves or not the system.

Step 5

If, as a result of the simulation in step 4, none of the patients leaves the system, then the simulation clock is advanced by $\Delta t$ time units and step 2 is repeated.

Step 6

If, as result of step 4, a patient leaves the system in the interval $\left( {t, t +\Delta t} \right)$, then the patient is discharged at a time uniformly distributed between t and $t +\Delta t$. The state of the system is updated and the simulation clock is advanced by $\Delta t$. Repeat step 2.

Each time the simulation clock is advanced, the accelerated time in system of a patient is updated according to expression:

$$t_{e} \leftarrow t_{e} + (t - t_{e} )\frac{{\mu_{i} }}{\mu }$$

These results allow us to use discrete-event simulation to simulate the G/G/c/c queuing model with varying service times dependent on the number of occupied beds.

For example, in the case of the Weibull distribution with parameters α and β, a patient’s probability of leaving the health system during the next $\Delta \varvec{t}$ time interval, if the current bed level occupancy is i, is calculated using the hazard function $h_{{T_{{\mu_{i} }} }} \left( t \right) = \frac{{\mu_{i} }}{\mu }\frac{\alpha }{\beta }\left( {\frac{{\mu_{i} }}{\mu \beta }t} \right)^{\alpha - 1}$.

Of course, the smaller step $\Delta \varvec{t}$, the greater the accuracy of the approximation, but also the greater computational effort because more steps are necessary to progress through time. Thus a compromise solution has to be adopted to balance accuracy and computational effort.

We have tested this method with the exponential distribution and obtained very good results taking 10 min-steps for average LoS = 1 day. Nevertheless, the accuracy of the approximation in (9) not only depends on the size of $\Delta \varvec{t}$, but also on the shape of the hazard function $h\left( t \right)$. When $h\left( t \right)$ is upper bounded, the error term in approximation (1) in (9) is proportional to $(\Delta t)^{2}$. This is the case for the usual distribution functions used to model LoS (loglogistic, lognormal and phase-type). When $h\left( t \right)$ increases as a linear function ($h\left( t \right) \sim ct )$ then, the error term is proportional to $t\Delta t$. This is the case for Weibull distribution with shape parameter $a = 2$. Exponential increments in the hazard function produces large error terms, as for example in the Smallest Extreme Value distribution. However, this type of distribution has not been reported on the statistical modelling of LoS studies. The approximation error in trapezoidal method depends on $h^{\prime \prime } \left( t \right)$, the second derivative in the interval $\left( {t, t +\Delta t} \right)$:

$$\left| {error} \right| < k \frac{{(\Delta t)^{3} }}{48} ,\;{\text{under}}\;{\text{the}}\;{\text{assumption}}\;\left| {h^{\prime \prime } \left( t \right)} \right| < k.$$

We conduct a computational study to analyze the stability of the estimations of the rejection probability depending on the size of parameter $\Delta t$. Table 5 shows the results for different distribution functions and considering the optimal service rates for a rejection value of 4 % for the M/M/5/5 model, with λ = 4 and $\mu$ = 1 (see Table 1). The distributions considered are: Exponential (with both Markovian and non-Markovian arrivals), Weibull with shape parameter α = 5, Loglogistic with scale parameters σ = 0.1 and σ = 0.6, Smallest Extreme Value distribution with scale parameters σ = 1.1 and σ = 1.5. Six values for parameter $\Delta t$ are analysed (60, 30, 20, 10, 5 and 1 min), representing 4.2, 2.1, 1.39, 0.69, 0.35 and 0.069 % of the expected LoS, respectively. The simulations were extended to attain a confidence interval with precision ±0.01. A rapid convergence in the estimation and a small difference between the values obtained for $\Delta t$ = 10, 5, 1 is observed for the Weibull and Loglogistic distributions. Nevertheless, according to above comments, the Smallest Extreme Value distribution presents a slower convergence.

Table 5 Estimations of the rejection probability with different sizes of parameter ∆t

3.3 Results for the general model

The methodology presented in Sects. 3.1 and 3.2 can be used to find optimal management policies in any general ICU model including elective patients and more realistic LoS distributions. As an example, we have chosen a 5-bed ICU (M/G/5/5 queuing model) and several distributions for modelling the patient LoS: Weibull, Loglogistic and Phase-type distributions. The Weibull distribution with shape parameter α = 1 provides the exponential distribution, when α > 1 (α < 1) the hazard function is monotonically increasing (decreasing). As the shape parameter becomes closer to 1, the Weibull distribution gets closer to the exponential distribution. Loglogistic distribution, as the lognormal, is frequently used to model LoS. Its hazard function reaches a maximum and then decreases asymptotically to zero. The Phase-type distributions allow for a more realistic modelling of the LoS of a patient by associating each state of the distribution to a different health status. The influence of the distribution of the arrival process is also considered by mixing an arrival Poisson process for outpatients with a Deterministic arrival process for elective patients (G/G/5/5 queuing model).

Modelling the LoS by a phase-type distribution can prevent from discharge a patient not sufficiently recovered. In this study, we consider a phase-type distribution with three states. When a patient enters to the ICU, the Markov chain is in state 1, representing a very bad health condition. States 2 and 3 are used to represent better health status, in such a way that a patient can only be discharged from state 3. To model the clinical worsening of the patient’s health (due to infections, for example) the transition from upper to lower states is allowed (Fig. 5).

Fig. 5

Phase-type distribution diagram

Control problems (GP1) to (GP4) are solved by using optimization with simulation and the method presented in Sect. 3.2. Tables 6, 7, 8 show the results for several M/G/5/5 queuing models, considering different LoS distributions (Weibull with shape parameter α = 5, Loglogistic with scale parameters σ = 0.2 and 3-phase-type distribution with q2 = 0.2 and q3 = 0.15). We study the phase-type distribution model allowing the shortening of a patient stay in both any phase-state (case 1) and only in state 3 (case 2), a more realistic situation. G/G/5/5 queuing model is also analysed by mixing a Poisson process with rate arrival $\lambda = 2$ and 2 elective patients per day. This general arrival process is considered for both exponential and Weibull (α = 5) LoS distributions. The parameters of the distributions has been adjusted to provide a distribution with mean $1 = E\left( G \right) = 1/\mu$. In all cases the arrival rate is constant, $\lambda = 4$. The Erlang Loss expression, provides the probability of a full ICU, $P_{c}^{*} = 0.199067$. The optimization problems are solved for percentage of rejected patients of $\varepsilon_{j}$ = 16, 10, and 4 %.

Table 6 Aggressive policies: Optimal management policies for (GP3) and (GP4) control problems, for different G/G/5/5 models and percentage of rejected patients $\varepsilon_{j}$ = 16, 10, and 4 %

Table 7 Equitable policies: Optimal management policies for (GP1) control problem, for different G/G/5/5 models and percentage of rejected patients $\varepsilon_{j}$ = 16, 10, and 4 %

Table 8 Cautious policies: Optimal management policies for (GP2) control problem, for different G/G/5/5 models and percentage of rejected patients $\varepsilon_{j}$ = 16, 10, and 4 %

An important conclusion is that the same structure of solutions is obtained for the general case than for the Markovian case. Aggressive, equitable and cautious policies are obtained as solutions to the optimization problems. Furthermore, the values that determine the service rates are very similar to those provided by the exponential distribution (see Table 1) for all the Markovian arrival models but the Phase-type distribution (case 2). Results for the non-Markovian arrival case also differ from the M/M/5/5 model.

4 Conclusions and final remarks

In this paper we have formulated a new control problem to represent the triage process that many physicians frequently face: the allocation of the available health resources in the best way. We focus on intensive care units, where rejection of incoming patients has to be balanced with the early discharge of current patients.

Control problems (P1) to (P4) are not convex optimization problems. By using a convex representation of the posynomials included in the expressions (6) and (7), the problems turn out to be the maximization of a convex function over a set. Such problems are usually hard to solve. Queuing theory and classical nonlinear optimization methods provide the solution to the control problem in the Markovian case while a simulation based optimization method has been developed to deal with the general case.

An interesting contribution is the interpretation of the three types of solutions obtained from the optimization problems when different expressions for the shortening of the LoS are considered. Physicians of the ICU understood them and identified the cautious type solution as representative of the way to deal with the pressure of a near-full ICU in the real setting.

Through a broad computational analysis we can conclude that the three management policy structures emerge as solution of the control problems independently of the LoS distribution, the size of the ICU, the occupancy ratio and the arrival pattern.

Experimental results show the capability of the proposed methodology to deal with optimization problems posed on real size ICUs without computational issues. Studies analyzing the size of hospital’s ICUs conclude that 8–12 beds are considered as the optimum [see for example Vallentin and Ferdinandi (2011)] while descriptive analysis indicates that the average number of beds is 17, 20 and 12 in cardiac, medical/surgical and pediatric intensive care units, respectively [Leleu et al. (2012)].

The recovery status of the patient has been modeled by using the states of a phase-type distribution to prevent the early discharge of a patient which has not recovered sufficiently.

The model presented in this paper can be readily extended to include different types of patients. In practice patients are clustered in different groups according to the type of illness, origin, necessary treatment, etc. Each group is characterized by its own arrival pattern and stochastic LoS.

Nevertheless, in order to make this theoretical analysis useful for the effective control of the health care system, it is necessary to take further steps in the analysis of the solution: physicians need flexible and medically-meaningful operative rules for shortening the LoS of a patient to the degree that will result in the service rates dictated by the theoretical analysis. The discussion of how the theoretical solutions can be transformed into effective management rules to guide doctors’ decisions constitutes our work in progress.