Modelling and analysis of healthcare inventory management systems

Abstract

The core competency of the healthcare system is to provide treatment and care to the patient. The prime focus has always been towards appointing specialized physicians, well-trained nurses and medical staffs, well-established infrastructure with advanced medical equipment, and good quality pharmacy items. But, of late, the focus is driven towards management side of healthcare systems which include proper capacity planning, optimal resource allocation, and utilization, effective and efficient inventory management, accurate demand forecasting, proper scheduling, etc. and may be dealt with a number of operations research tools and techniques. In this paper, a Markov decision process inventory model is developed for a hospital pharmacy considering the information of bed occupancy in the hospital. One of the major findings of this research is the significant reduction in the inventory level and total inventory cost of pharmacy items when the demand for the items is considered to be correlated with the number of beds of each type occupied by the patients in the healthcare system. It is observed that around 53.8% of inventory cost is reduced when the bed occupancy state is acute care, 63.9% when it is rehabilitative care, and 55.4% when long-term care. This may help and support the healthcare managers in better functioning of the overall healthcare system.

1 Introduction

The health expenditure per capita is increasing year after year and the cause of the rise in health expenditure is due to the aging population growth combined with new and more expensive treatments and pharmacy items (World Health Statistics [1]). Furthermore, the factors that may add to these inefficiencies are the limited resources in a healthcare system, a mismatch in demand and supply of pharmacy items, and inappropriate inventory control systems (Nicholson et al. [2]). Therefore, the healthcare sector is prone to advance in effective and efficient inventory management techniques under the stochastic nature of the demand for pharmacy items as best as possible. Nonetheless, due to the demand and supply uncertainties and variability, the healthcare inventory systems seem to be quite complex (Saedi et al. [3]). Adding to the complication are the heterogeneity among patients and their bed occupancy aspects (McClean et al. [4]).

The ultimate objective of healthcare organization is to provide maximum service to the patients; hence certain steps or processes may be prepared prior to the arrival of the patient, such as the pharmacy items are inventoried in a hospital pharmacy so that they are available to the patients when needed along with other hospital services for proper treatment and care. However, the main problem is that the healthcare systems have limited space and budget to be spent on inventory items, and a number of pharmacy items have a limited shelf-life. In addition, there is constant pressure on reduction in healthcare costs. It is observed that in a healthcare system, pharmacy items represent 70–80% of the supply costs (Volland et al. [5]). Hence, an effective and efficient inventory control systems for hospital pharmacy is desired.

In contrast to the existing studies, this study focuses on inventory control systems for hospitals that utilize the information of bed occupancy. The daily requirements of the pharmacy items in a hospital vary according to the number of different types of beds occupied by patients and their length-of-stay in each bed. Hence, the model considers the impact of bed occupancy in hospitals on total inventory costs of hospital items under various constraints and conditions. To the best of our knowledge, the consideration of bed occupancy on inventory management of healthcare systems is a novel approach.

In this paper, the inventory model considering impact of bed occupancy is formulated as a Markov Decision Process (MDP), and solved by backward induction algorithm. The number of beds occupied by the patients in the system of each bed type and the inventory level of a pharmacy item is considered as the two MDP states, and an optimal state- and time-dependent inventory control system is determined. In this model, the number of beds occupied by the patients in the system of each bed type is correlated with the demand for pharmacy items which is stochastic in nature. The proposed approach in managing inventory in hospital based on routinely collected large-scale actual data is essential to facilitate managing and analysis of the hospital inventory system. Therefore, a case study in a multispecialty hospital located in Kolkata, India is considered to collect real data and validate the proposed model. The inventory-related decisions through the proposed model will significantly help the hospital managers, staff, medical personnel and policy-makers to achieve a systematic approach of optimal usage of scarce healthcare resources.

The organization of the paper is as follows. Section 2 reviews the relevant literature on healthcare inventory management systems. Section 3 presents a methodology on modelling and analysis of inventory control systems of hospital pharmacy items considering bed occupancy in hospitals. The proposed model is validated through a case study in a multispecialty hospital located in Kolkata in Sect. 4, and interesting results and findings on comparing the traditional model with the proposed model are discussed in Sect. 5. Finally, the conclusions and possible future research avenues are discussed in Sect. 6.

2 Review of literature

In recent times, there has been an increasing number of studies on healthcare inventory management systems. The problem aspects considered in the existing literature are variability in demand and supply of items, replenishment policies, limited shelf life, and finite/infinite planning horizon. It is observed that although the nature of demand and supply in the healthcare setting is stochastic, a limited number of papers consider this aspect (Nicholson et al. [2], Saedi et al. [3], Vila-Parrish et al. [6]). Apart from that, the common replenishment policies considered are continuous review policy with parameters: order quantity and reorder point (Roni et al. [7]), and periodic review policy with parameters: review period and maximum inventory level (Gebicki et al. [8]). Furthermore, the different types of healthcare inventory models are classified based on the objective functions, decision variables, and constraints and it is observed that most of the literature considered cost model i.e. minimizing inventory-related cost with service level constraints (Nicholson et al. [2], Guerrero et al. [9]; Uthayakumar and Priyan [10]), and a few papers focus on service model (Bijvank and Vis [11], Little and Coughlan [12]).

Operations research (OR) models may very well explain the healthcare inventory systems by formulating the model with the objective function to identify the optimal inventory parameters under certain constraints (Kelle et al. [13], Uthayakumar and Priyan [10]). Roni et al. [7] applied an optimization model by formulating a mixed integer programming in a non-linear environment with the objective of minimizing cost under various constraints. To represent the stochastic and complex nature in a healthcare setting, advanced OR models like Markov decision process (MDP) model is highly encouraged (Bijvank and Vis [11], Saedi et al. [3], Vila-Parrish et al. [6]). A semi-MDP model is developed by Rosales et al. [14] to analyze the continuous review two-bin inventory system. Haijema [15] considered the single type of patient but used an infinite horizon MDP model to find the optimal ordering, issuing and disposal policy that is solved by the value iteration method. Apart from that network flow analysis is suggested by Hovav and Tsadikovich [16] which considers a constrained cost minimization problem from the perspective of the healthcare organization. Since, the healthcare organization is assumed to consist of several distribution centers and a service provider, a network flow model is used to represent the problem. To model the performance of the inventory system in a hospital in case of inventory pooling, the whole process from drug demand to supply involved by patients, hospitals, distributors, and manufacturers in the traditional inventory mode and inventory pooling mode (Wu et al. [17]). Wang et al. [18] implemented the dynamic drum-buffer-rope (DDBR) replenishment model using a system dynamics approach. The limitations of mathematical programming are that they are not so well in predicting the operational performance under the pressure of real-world day-to-day variability, such as demand, transportation delays, production lead times, etc. Hence, simulation is used to test the inventory strategy chosen by the optimization model to test it further. Event-driven simulation allows the necessary flexibility in modeling (Gebicki et al. [8]), to evaluate the performance of different inventory systems based on the total cost and number of stockouts. Attanayake et al. [19] also tested their stochastic demand and lead time distributions inventory model by simulation. The outputs generated by simulation may be further used for statistical analysis. Apart from that, the greedy algorithm heuristics is also applied to solve the healthcare multi-echelon inventory problem by Nicholson et al. [2] for finding optimal par levels.

Hence, from the extensive review of the literature, it is observed that there is a lack of studies on inventory control systems at the hospital-level and focusing more at the patient level. There is a need to use improved information to understand the link between the number of beds of different types occupied by the patients with the demand for hospital inventory items. The development of real-time demand forecasting and inventory management systems that consider patient and bed occupancy information as an input is an opportunity for research. The development of advanced OR models that integrate these factors may be helpful for hospitals to understand and control their inventory-related costs, improve patient care and efficiently use limited hospital resources. In this paper, an optimal inventory control system is developed that incorporates the impact of patient bed occupancy on the daily demand of pharmacy items. The aim is to determine optimal state-dependent inventory control policy that minimizes expected total inventory-related cost subject to a number of constraints, such as inventory balance, space, and service level. The proposed model is further validated using the data from a multispecialty hospital located in Kolkata, India.

3 Methodology

The patients arriving and staying in a healthcare facility may be at various stages of care and treatment, such as acute care, rehabilitative care, and long-term care. Acute care patients occupy the hospital bed for a shorter period of time. Rehabilitative care patients occupy the hospital bed for a moderate period of time, while long-term care patients occupy the bed for a longer period of time (for example, geriatric patients). Thus, based on the length-of-stay of an individual patient, a bed is changed from one type to another. Furthermore, it is considered that there exists a correlation between the number of beds of each type occupied by patients (bed occupancy) and the demand for pharmacy items which changes stochastically. Considering the bed occupancy-dependent demand model, the inventory control system of the pharmacy items is modeled using the MDP approach and compared with the traditional inventory control approach in the hospitals. The objective of the study is to minimize total inventory costs in the system. The pharmacist in a hospital system is engaged with the problem of balancing among different inventor-related costs, such as ordering costs, holding costs, shortage costs, and expiration costs. Ordering costs includes the costs required for the processing of an order. The holding cost is incurred on the inventory on-hand at each time period. In case of stock-outs, shortage cost is incurred. Finally, each unit of medicine expired in the hospital pharmacy incurs an expiration cost.

The proposed model is based on the following assumptions and notations (Table 1).

Table 1 Model notations

Assumption 1

The inventory system under consideration includes inpatient pharmacy items. The outpatient pharmacy department is separate and is not considered.

Assumption 2

The pharmaceutical supplier is considered to be of infinite capacity and have a constant lead time.

Assumption 3

The planning horizon is finite as the pharmaceutical items expired or outdated after a fixed time period.

Assumption 4

Shortages are fulfilled by emergency order (other hospitals or suppliers) ensuring 100% service level.

Assumption 5

The quantity ordered arrives before any demand is realized.

Assumption 6

States of the system at the next decision epoch is not known in advance (since demand is random).

3.1 Traditional approach (Model-1)

The traditional inventory control approach is to determine the reorder level and order-up-to level (Gebicki et al. [8]). The reorder level is based on the demand during the lead time following a normal distribution and is expressed in Eq. 1.

$$R_{{S_{i} }} = \mu_{{S_{i} }} L + z_{{S_{i} }} \sigma_{{S_{i} }} \sqrt L$$

(1)

where $S_{i}$ is the type of bed occupied, $L$ is the constant lead time, $\mu_{{S_{i} }}$ is the average daily demand of item in state $S_{i}$, $\sigma_{{S_{i} }}$ is the daily demand standard deviation of an item in state $S_{i}$, and $z_{{S_{i} }}$ is the z-value of a standard normal distribution with respect to the service level assigned to each state, $S_{i} .$ The term $z_{{S_{i} }} \sigma_{{S_{i} }} \sqrt L$ is the safety stock considered to deal with the variability in demand. The order-up-to level is calculated as the sum of reorder level and economic order quantity, $u_{{S_{i} }}$ which is calculated as Eq. 2 (Hillier and Lieberman [20]).

$$u_{{S_{i} }} = \sqrt {\frac{{2m_{{S_{i} }} A_{{S_{i} }} }}{{h_{{S_{i} }} c_{{S_{i} }} }}}$$

(2)

where $A_{{S_{i} }}$ is the fixed ordering cost, $h_{{S_{i} }}$ is the percentage used for holding cost, $h_{{S_{i} }}^{\prime }$ is the shortage cost per unit, and $c_{{S_{i} }}$ is the purchase cost per unit.

3.2 MDP approach (Model-2)

The MDP model consists of five basic components, viz. the decision epochs, states, actions, transition probabilities, and cost functions and is described below.

3.2.1 Decision epochs

The time at which the decision is made is the decision epoch. Since the hospital inventory is checked every day, the decisions whether to place an order or not is taken once a day with finite planning horizon denoted by $T$ periods. Thus, a hospital inventory system makes ordering decisions at the beginning of every period over a finite time horizon.

3.2.2 States

The MDP model is described with two states: (i) the number of beds of each type occupied by patients (bed occupancy) and (ii) inventory level of a pharmacy item. The state of bed occupancy, $S_{i}$ is correlated with the pharmacy item demand process, $m_{t}$ and is modelled as a Markovian demand process. The inventory level is $x_{t}$ at the time period $t$.

3.2.3 Actions

At the beginning of period $t$, the order of quantity, $u_{t} \ge 0$ is placed with the knowledge that the current states are $\left( {S_{i} , x_{t} } \right)$ and demand is $m_{t}$. The order quantity $u_{t}$ will be delivered at the end of period $t$. In case the demand is more than the on-hand inventory after the order is received (i.e.$x_{t} + u_{t} < m_{t}$), an emergency order is placed to meet the demand, and the next period starts with zero on-hand inventory.

3.2.4 Transition probabilities

The probability that the type of bed changes from being occupied by a patient in a state $S_{i}$ to a patient in a state $S_{i + 1}$ (see Fig. 1) is denoted by the transition probability matrix, $P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right)$ (McClean et al. [4]). In addition, there is an absorbing state, $S_{n + 1}$ representing the discharge or death of the patient. If the patient is discharged or die from bed $S_{i}$ then the bed changes from being occupied by a patient in state $S_{i}$ to being occupied by a patient in state $S_{1}$ i.e. the initial state. It is assumed that all patients are initially admitted to state $S_{1}$, (for example, acute care state).

$$P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right) = \left[ {\begin{array}{*{20}c} {1 - p_{1} } & {p_{1} } & 0 & 0 & \cdot & \cdot & 0 \\ {q_{2} } & {1 - p_{2} - q_{2} } & {p_{2} } & 0 & \cdot & \cdot & 0 \\ {q_{3} } & 0 & {1 - p_{2} - q_{3} } & {p_{3} } & \cdot & \cdot & 0 \\ \cdot & \cdot & \cdot & \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot & \cdot & \cdot & \cdot \\ {q_{k} } & 0 & 0 & 0 & \cdot & \cdot & {1 - q_{n} } \\ \end{array} } \right]$$

For $i = 1, 2, \ldots , n$, $p_{i} = P\left\{ {patient\;in\;state\;S_{i + 1} \;at\;time\;t + 1|patient\;in\;state\;S_{i} \;at\;time\;t} \right\}$ and $q_{i} = P\left\{ {patient\;in\;state\;S_{n + 1} \;at\;time\;t + 1|patient\;in\;state\;S_{i} \;at\;time\;t} \right\}$. Let $b_{{S_{i,t} }}$ be the number of beds occupied in state, $S_{i}$ at time, $t$.

Fig. 1

A Markov chain showing changes in types of bed occupied by patients

Furthermore, the transition probability of the inventory level depends on the new demand and the action taken. The next inventory level state is defined in Eq. 3.

$$x_{t + 1} = x_{t} + u_{t} - b_{{S_{i,t} }} m_{t}$$

(3)

The demand process, $m_{t} = \left\{ {D_{t} ; t \ge 1} \right\}$ is modulated by the state, $S_{i}$ and defined as $p\left( {m_{t} = D_{t} |S_{i} } \right)$ i.e. the probability that demand is a discrete value, $m$ for state, $S_{i}$. Thus, the transition probability matrix of the inventory level state is denoted in Eq. 4.

$$P\left( {x_{t + 1} \left| {x_{t} } \right.,u_{t} } \right) = \left\{ {\begin{array}{*{20}l} {p\left( {m_{t} = D_{t} |S_{i} } \right),} \hfill & {if\quad x_{t + 1} = x_{t} + u_{t} - b_{{S_{i,t} }} m_{t} } \hfill \\ {0,} \hfill & {otherwise} \hfill \\ \end{array} } \right.$$

(4)

Thus, the probability of reaching the next inventory level state depends only on the current inventory level state,$x_{t}$ the current bed occupancy modulated demand process, $m_{t}$ and the current action, $u_{t}$ and not on the history of past states and actions (following Markovian property).

3.2.5 Cost functions

Cost is incurred while taking action in a given state. For instance, depending on the state, $S_{i}$, and the number of beds occupied by patients in each state, $b_{{S_{i,t} }}$, the total inventory cost function is developed considering the purchase cost of a pharmaceutical item $\left( {c_{{S_{i} }} } \right)$, the ordering cost per unit ordered $\left( {A_{{S_{i} }} } \right)$, the percentage used for holding cost per unit per period $\left( {h_{{S_{i} }} } \right)$, and the shortage cost per unit per period $\left( {sh_{{S_{i} }} } \right)$ for each unsatisfied demand. Both holding and shortage costs are incurred at the end of the period. The unsatisfied demand is immediately met by emergency order which is included in the shortage cost. An expiration or outdating cost $\left( {ex_{T } } \right)$ is incurred for at the end of the planning horizon, $T$. Thus, the cost function depending on the action and states is given in Eqs. 5 and 6.

$$TC\left( {S_{i,t} ,x_{t} ,u_{t} } \right) = A_{{S_{i} }} \delta \left( {u_{t} } \right) + u_{t} c_{{S_{i} }} + \mathop \int \limits_{0}^{{x_{t} }} h_{{S_{i} }} \cdot c_{{S_{i} }} \left( {x_{t} + u_{t} - b_{{S_{i,t} }} m_{t} } \right)f\left( m \right)dm + \mathop \int \limits_{{x_{t} }}^{\infty } h_{{S_{i} }}^{\prime } \left( {b_{{S_{i,t} }} m_{t} - x_{t} - u_{t} } \right)f\left( m \right)dm$$

(5)

$$TC_{T} \left( {S_{i,T} ,x_{T} } \right) = ex_{T} \cdot x_{T}$$

(6)

Since, the demand for the item is random, stochastic and depends on the changing number of beds occupied of each type, the decisions are based on the expected cost which is given in Eq. 7.

$$TC\left( {S_{i,t} ,x_{t} ,u_{t} } \right) = \mathop \sum \limits_{i = 1}^{n} P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right)\mathop \int \limits_{0}^{\infty } P\left( {x_{t + 1} \left| {x_{t} } \right.,u_{t} } \right)TC\left( {S_{i,t} ,x_{t} ,u_{t} } \right)f\left( m \right)dm$$

(7)

Subject to

$$x_{t + 1} = x_{t} + u_{t} - b_{{S_{i,t} }} m_{t}$$

(8)

$$u_{t} \le CAP - x_{t}$$

(9)

$$Prob\left( {x_{t} + u_{t} \le m_{t} } \right) \le \beta$$

(10)

$$u_{t} \ge 0$$

(11)

$$0 \le t \le T - 1$$

(12)

The constraint in Eq. 8 is the inventory balance equation, Constraint in Eq. 9 is the limited space capacity equation where $CAP$ denotes the limited number of inventory items that can be stocked. Constraint in Eq. 10 is the service level constraint where, the probability that the demand is met by the available on-hand inventory after the order is received is limited, and $0 < \beta < 1$. Constraint in Eq. 11 is the positivity constraint of the order quantity, and Constraint in Eq. 12 is the finite planning horizon constraint.

3.2.6 Optimal decision policy

The optimal decision policy is defined by the set of decision rules at every decision epochs that decides what action to take for any given state and any given decision epoch. Hence, the policy is determined by the order quantity that gives the minimum expected total inventory cost over the length of the planning horizon. The total cost of a policy over a finite planning horizon of length, $T$ and starting at state $\left( {S_{i,t} ,x_{t} } \right)$ is given in Eq. 13.

$$C_{T} \left( {S_{i,t} ,x_{t} ,u_{t} } \right) = E\left( {\mathop \sum \limits_{t = 1}^{T - 1} TC_{t} \left( {S_{i,t} ,x_{t} ,u_{t} } \right) + TC_{T} \left( {S_{i,T} ,x_{T} } \right)} \right)$$

(13)

The optimization problem is solved by backward induction i.e. it starts with the last time period where the decisions need to be taken (in this case, $T - 1$), and moves to time period $t = 1$. Thus, the total cost incurred from period $t$ onward is defined by Bellman’s optimality equations (Puterman [21]) in Eq. 14.

$$C_{t}^{*} \left( {S_{i,t} ,x_{t} ,u_{t} } \right) = \mathop {\hbox{min} }\limits_{{0 \le u_{t} \le CAP}} \left( {TC_{t} \left( {S_{i,t} ,x_{t} ,u_{t} } \right) + \mathop \sum \limits_{i = 1}^{n} P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right)\mathop \int \limits_{0}^{\infty } P\left( {x_{t + 1} \left| {x_{t} } \right.,u_{t} } \right)C_{t + 1}^{*} \left( {S_{i,t + 1} ,x_{t + 1} ,u_{t + 1} } \right)f\left( m \right)dm} \right)$$

(14)

The optimal decision at any decision epoch and for any given state is the action i.e. the order quantity, $u_{t}$ as given in Eq. 15.

$$u_{t}^{*} \in arg\mathop {\hbox{min} }\limits_{{0 \le u_{t} \le CAP}} \left( {TC_{t} \left( {S_{i,t} ,x_{t} ,u_{t} } \right) + \mathop \sum \limits_{i = 1}^{n} P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right)\mathop \int \limits_{0}^{\infty } P\left( {x_{t + 1} \left| {x_{t} } \right.,u_{t} } \right)C_{t + 1}^{*} \left( {S_{i,t + 1} ,x_{t + 1} ,u_{t + 1} } \right)f\left( m \right)dm} \right)$$

(15)

Thus, the cost function $C_{t}^{*} \left( {S_{i,t} ,x_{t} ,u_{t} } \right)$ is the minimum expected total inventory cost over the planning horizon given that the system starts in states $\left( {S_{i,t} ,x_{t} } \right)$. The optimal order quantity is $u_{t}^{*}$ for each state and each decision epoch.

To validate the proposed model, a case study is considered and is described in the next section.

4 Case study

The data is collected from a multispecialty hospital in Kolkata, India. The hospital has 15 departments, 24 types of care units with a total of 226 functional beds, more than 10,000 patients in a year, 500 physicians and approximately 7000 pharmacy items stored in a hospital pharmacy. The purchasing department makes inventory-related decisions with the consultation of pharmacists. The pharmacy items maintained are based on the physician preference and as such no standardization of items is maintained in the hospital. The pharmacy meets the demand of individual patient as per the prescription or medication order. The pharmacy places order to the pharmaceutical supplier according to the replenishment decisions. The current inventory control system in the hospital under study follows a traditional approach for all pharmacy items with the control parameters (reorder level and order-up-to level) decided purely on the basis of historical data on annual consumption. The lead time of items distribution from the suppliers is generally constant.

To validate the proposed model, the following data are collected from the hospital information system: patient ID and name, patient arrival and discharge dates, pharmacy item code, and name, the quantity of pharmacy items issued to patients, and unit price of these items. Around 11,772 data of patients admitted in the hospital and consuming more than 5000 types of pharmaceutical items during the year 2015–2016 were collected. The format in which the inventory data is stored in the hospital database is shown in Table 2.

Table 2 Format for storing hospital data

The variability in bed occupancy and demand of pharmacy items is shown in Fig. 2. It is interesting to find that the correlation between the number of beds occupied by the patients in the hospital and demand for pharmacy items is very high (84%). Hence, the stochastic demand function captures the patient information in the form of bed occupancy.

Fig. 2

Variability in patient information and patient demand

Furthermore, the length-of-stay in days (LOS) is calculated by subtracting the arrival and discharged dates. It is observed that the beds occupied in a year for a short time $\left( {0 \le LOS \le 7\;days} \right)$ are more compared to the number of beds occupied for higher LOS $\left( {8 \le LOS \le 20\;and\;LOS\; > 20\;days} \right)$ (Fig. 3). These three different groups of LOS are termed as acute, rehabilitative and long-term and are the Markov states (see Fig. 4) as they follow the Markovian property (see Appendix).

Fig. 3

Variation in the number of beds occupied and length-of-stay

Fig. 4

A Markov chain showing changes in three types of bed occupied by patients

It is assumed that all patients are initially admitted as an acute patient. The transition probability matrix describes the probability with which one state changes to another with time and is computed from the LOS data. The transition probabilities of the bed occupancy states are derived from the real data and are presented in Table 3. Since it is unlikely that the bed occupied by patients in acute care bed (State-1) will be in the long-term care bed (State-3) in the next day, therefore, $P\left( {S_{3, t + 1} \left| {S_{1,t} } \right.} \right) = 0$. Similarly, it is very unlikely that bed occupied by patients in long-term care bed (State-3) will be occupied by rehabilitative care patients (State-2) in the next day $\left( {i.e. P\left( {S_{2, t + 1} \left| {S_{3,t} } \right.} \right) = 0} \right)$. However, it is highly likely for the bed occupied by patients in an acute care bed (State-1) and long-term care bed (State-3) to be in the same state in the next day $\left( {i.e. P\left( {S_{1, t + 1} \left| {S_{1,t} } \right.} \right) = 0.941\;and\;P\left( {S_{3, t + 1} \left| {S_{3,t} } \right.} \right) = 0.968} \right)$. Furthermore, beds occupied by patients in rehabilitative care (State-2) may change to long-term care bed (State-3) in the next day with probability 0.035 if the patients are not discharged, and may change to acute care bed with probability 0.225 if the patients are discharged.

Table 3 Transition probability matrix, $P\left( {S_{i, t + 1} \left| {S_{i,t} } \right.} \right)$

Apart from that, each patient’s requirement for pharmacy items varies daily according to their states, hence the amount required of each item depends on the number of beds occupied in each state. A particular pharmacy item named Intravenous Cannula (Venflon) is considered for the study. The descriptive statistics summary of the state-dependent demand for the selected pharmacy item is given in Table 4. The demand distribution of this item for three different states follows normal distribution and is shown in Fig. 5.

Table 4 Summary of state-dependent demand for the selected pharmacy item

Fig. 5

Demand distribution of a pharmacy item named Intravenous Cannula (Venflon)

Furthermore, variation between the number of beds occupied by patients in a hospital and demand for the pharmacy items exists and is shown in Fig. 6 for a sample of data.

Fig. 6

Daily demand versus number of beds occupied by patients in a hospital

As per the recommendations from the hospital administration, the following values of the input parameters are considered. The purchase cost of the item, $c_{{S_{i} }} = Rs 110$, ordering cost per unit ordered, $A_{{S_{i} }} = Rs 10$, the percentage used for holding cost per unit per period, $h_{{S_{i} }} = 0.11$, the shortage cost per unit per period, $h_{{S_{i} }}^{\prime } = Rs 100$, and expiration or outdating cost, $ex_{T } = Rs 200$. The limited capacity, $CAP = 100$ items, service levels $\left( \beta \right)$ are 99.9%, 99% and 97.5%, and the lead time $\left( L \right)$ is 1 day.

5 Results and discussions

The optimal values of the reorder point and order-up-to level are obtained for the selected pharmacy item and are illustrated in Table 5 for both the models (traditional and MDP). It is observed that the inventory policy obtained by the traditional approach can be characterized as min–max policy i.e. $\left( {s_{{S_{i} }}^{\prime } , S_{{S_{i} }}^{\prime } } \right)$ policy, whereas, the inventory policy obtained by the MDP approach is characterized as state-dependent base stock policy, i.e. $\left( {s_{{S_{i} }}^{\prime } - 1,S_{{S_{i} }}^{\prime } } \right)$ policy. The $\left( {s_{{S_{i} }}^{\prime } , S_{{S_{i} }}^{\prime } } \right)$ policy denotes that as the inventory level of the item at state, $S_{i}$ is at or below the reorder level, $s_{{S_{i} }}^{\prime }$, an order is placed to update the inventory level to order-up-to level, $s_{{S_{i} }}^{\prime }$. The $\left( {s_{{S_{i} }}^{\prime } - 1,S_{{S_{i} }}^{\prime } } \right)$ policy denotes that as the inventory level is at or below the reorder level, $s_{{S_{i} }}^{\prime } - 1$, an order is placed to update the inventory level to order-up-to level, $s_{{S_{i} }}^{\prime }$.

Table 5 State-dependent optimal inventory control parameters (reorder level and order-up-to level)

Furthermore, it is observed that there is a significant reduction in order-up-to levels obtained from the MDP approach (Model-2) compared to the traditional approach (Model-1) (Fig. 7). Apart from that, it may be observed from Fig. 7 that the values of reorder level and order-up-to level decreases with a decrease in service level for each state.

Fig. 7

Variation of reorder point and order-up-to level with states and service levels

The results in Fig. 8 indicates that the expected total inventory cost decreases as the time period tends to move towards the end of the planning horizon (in this case, $T = 12$ days). In addition, the cost varies with the current inventory level. It decreases till a certain inventory level, and then increases, thus the optimal expected total inventory cost is achieved (see Fig. 9).

Fig. 8

Expected total inventory cost versus time periods with current inventory level of 25 items

Fig. 9

Expected total inventory cost versus inventory level

The total inventory cost obtained from Model-1 and Model-2 is compared (Fig. 10) by varying the current bed occupancy states. The results show significant cost reduction in the case of Model-2 compared to Model-1 for each state (53.8% when the current state is acute care, 63.9% when the current state is rehabilitative care, and 55.4% when long-term care).

Fig. 10

Comparison between total inventory costs of two different models

6 Concluding remarks and future scope of research

In this paper, an effective and efficient inventory control system for hospital pharmacy is developed considering the correlation between the daily requirements of the pharmacy items in a hospital and the number of different types of beds occupied by patients in a hospital. The objective is to minimize the total inventory costs under space and service level constraints and obtain the optimal control parameters (reorder level and order-up-to level). The number of beds occupied by the patients in the system of each bed type and the inventory level of a pharmacy item is considered as the two Markov states and an optimal state- and time-dependent inventory control system is determined. A case study in a multispecialty hospital located in Kolkata, India is considered to collect real data and validate the proposed model. It is observed that modeling the correlation between the demand of pharmacy items in a hospital and the number of different types of beds occupied by patients in a hospital as Markovian demand process has significantly reduced the order-up-to level and expected total inventory cost. Hence, the proposed inventory control systems will help the hospital managers, staff, medical personnel and policymakers to achieve a systematic approach of optimal usage of scarce healthcare resources. The future scope of the research may define the states to indicate patient information, such as medical condition, clinical diagnosis, and treatment stages. Moreover, in the present study, the probability distribution of lead time demand is obtained from the historical data; however, in many cases due to lack of information, knowledge of probability distribution of random variables are not known. In such cases, distribution-free approaches can be explored (Malik et al. [23]; Sarkar and Mahapatra [24]). Additionally, the return of excess inventories from the point-of-use locations to the hospital pharmacy may be considered (Cheikhrouhou et al. [25]), along with a multi-echelon inventory system (Pal et al. [26]). Since, many healthcare inventory items like injections and medicines are perishables and their quality may also deteriorate with time (Pal et al. [27]). Such situations may be considered in future healthcare inventory models. Overall, the focus should be to build a sustainable inventory model with affordable healthcare services (Dey et al. [28]; Sarkar et al. [29]).

Appendix

1.1 Testing markovian property

To prove that the states hold the Markov assumption, a higher-order Markov model is designed (Chen and Hong [22]). It is because if the Markov assumption holds then building memory into the model via higher order models should have no effect on the transition probabilities.

1.2 Hypotheses of interest and test statistics

Suppose bed states, $\left\{ {S_{t} } \right\}$ is a strictly stationary time series process. It follows a Markov process if the conditional probability distribution of $S_{t + 1}$ given the information set $Z_{t} = \left\{ {S_{t} , S_{t - 1} , \ldots } \right\}$ is the same as the conditional probability distribution of $S_{t + 1}$ given $S_{t}$ only.

This can be expressed by the null hypothesis,

$$H_{0} :P (S_{t + 1} \le i|Z_{t} ) = P (S_{t + 1} \le i|S_{t} )$$

for all $i$ and for all $t \ge 1$. Under $H_{0}$, the past information set $Z_{t - 1}$ is redundant i.e. the current state variable or vector $S_{t}$ will contain all information about the future behaviour of the process that is in the current information set $Z_{t}$.

The alternative hypothesis is when

$$H_{A} :P(S_{t + 1} \le i|Z_{t} ) \ne P(S_{t + 1} \le i|S_{t} )$$

for some $t \ge 1$, then $S_{t}$ is not a Markov process. The Chapman–Kolmogorov equation is able to detect Markovian property.