1 Introduction

Medical service is an important part of the social security system and the modern service industry. Currently, with the increasing demands of medical services and the extensive use of advanced medical equipment, patient scheduling problems in medical services with the aid of medical equipment have received more and more concerns (Meskens et al. 2013; Petrovic et al. 2006; Riff et al. 2016; Zhang et al. 2015; Zhong et al. 2014).

In recent years, a survey shows that the overall prevalence of chronic kidney disease is estimated to be 8–16% worldwide (Jha et al. 2013). Hemodialysis service, which targets the maintenance of the amount and composition of body fluid within a range that is close to normal, is an important way to treat chronic kidney disease (Tomoyuki et al. 2015). As a relative shortage of organs limits transplantation rates, the number of patients on dialysis is increasing (Power et al. 2009). Therefore, the patient scheduling in hemodialysis service is of great practical significance.

Hemodialysis service centers provide services for patients on dialysis by using hemodialysis equipment. After some examination and diagnosison of the patients, doctors develop hemodialysis treatment plans for them in which several treatments have to be done in a week or 2 weeks. Then, a treatment schedule will be made for all the patients according to these plans and the availability of devices. Patient scheduling in hemodialysis service is a mapping of patients to devices, shifts and spaces in these centers in a planning period. In patient scheduling, managers usually have to consider the homogeneity of the break time between two treatments for every patient. In these centers, dialysis devices may have a variety of functions, which could provide different types of dialysis treatment and are usually located in multiple rooms. In a treatment plan, a patient should be served for various types of dialysis treatment. A service for a patient on one shift is provided by only one device and one device can only serve for one patient on a shift. Additionally, the patients often have the preferences on time, device and space which often have to be satisfied as much as possible. The relationships among patients, devices, rooms and shifts are complicated which are different from other patient scheduling problems and timetabling problems on which surveys can be found in Ahmadi-Javid et al. (2017), Hulshof et al. (2012) and Hadidi (2015). These characteristics make patient scheduling in hemodialysis service very challenging.

Motivated by real needs from a hemodialysis service center in Wuhan China, this study aims to develop an optimization model for patient scheduling in hemodialysis service and propose some effective approaches with a synthetic objective on minimizing the gross cost for utilizing devices, the number of night treatments, satisfying the patients preference as much as possible and balancing devices utilization.

The remainder of the paper is organized as follows. Section 2 is a discussion of related work. Problem description and formulation of Patient Scheduling in Hemodialysis Service are in Sect. 3. In Sect. 4, hemodialysis service schedules in several levels are defined. A heuristic algorithm and a rollout algorithm are developed to solve the problem. The numerical experiments are presented in Sect. 5. The conclusions are summarized in Sect. 6.

2 Related work

As mentioned before, patient scheduling in hemodialysis service is a patient scheduling problem, and is also a timetabling problem. So, patient scheduling problems, timetabling problems and their approaches are briefly reviewed in the following. Prior research work on multi-function resource scheduling is also reviewed. Finally, comparison of this problem with the past is conducted.

As for hemodialysis scheduling problem, Holland (1994) compared two scheduling models of a hemodialysis service center considering device utilization, length of service and the capability of the dialysis unit. He pointed out that it was better to have flexible start time instead of fixed start time in daily schedule. Ogulata et al. (2009) indicated that physical therapies, hemodialysis and radiation oncology departments in which patients go through lengthy and periodic treatments need to utilize their limited and expensive equipment and human resources efficiently. Peña et al. (2013) studied inpatients dialysis scheduling problem for patients with end-stage renal disease. In the problem, each patient requires a different number of time slots to be treated and the dialysis devices are partitioned into blocks according to the needs of the patients. Yao et al. (2012) designed a heuristic algorithm focusing on the dialysis machines. The algorithm obtained schedule for each machine assuming uniform treatment break time to improve efficiency.

Patient scheduling is conducted to assign patients to the scarce hospital resources optimally with objectives on patients waiting time, satisfaction, completion time of clinics etc., in which outpatient scheduling has been paid most attention. In the latest literature review on outpatient scheduling, Ahmadi-Javid et al. (2017) discussed modeling approaches, solution methods and environmental factors (patient unpunctuality, physician lateness, interruption, patient no-show and cancellation, patient preference, random service time, patient heterogeneity, and type of appointment required by patients) in decisions of various levels of strategy, tactic and operation. The survey of Cayirli and Veral (2003) and Gupta and Denton (2008) can be also referred. In the work of recent years on patient scheduling, Saremi et al. (2015) addressed the challenges of scheduling patients with stochastic service times and heterogeneous service sequences in multi-stage facilities by proposing an optimization method termed multi-agent tabu search (MATS). Riise et al. (2016) considered a class of multi-mode appointment scheduling problems, with variable resource availability and resource setup times, and presented a novel exact method based on a recursive version of logic-based Benders decomposition. Azadeh et al. (2015) addressed a semi-online patient scheduling problem in a pathology laboratory and proposed a mixed integer linear programming model to maximize the level of patient satisfaction. Yan et al. (2015) proposed a sequential appointment scheduling method to balance the benefits of clinic and patients satisfaction considering patient choice and service fairness simultaneously. Condotta and Shakhlevich (2014) studied a multi-criteria optimization problem in the context of booking chemotherapy appointments, and proposed an integer linear programs based on the concept of a multi-level template schedule. Wu et al. (2014) studied a patient scheduling problem with periodic deteriorating maintenance that aims to minimize the number of tardy medical treatment of all the patients. Ogulata et al. (2009) applied a simulation approach to a patient scheduling for a university radiation oncology department that aims to minimize delays in treatments and to maintain efficient use of the daily treatment capacity.

From the problem structure point of view, patient scheduling in hemodialysis service scheduling problem is also similar to the well-studied educational timetabling problems which includes university course timetabling problem, examination timetabling problem and school timetabling problem (Hadidi 2015; Pillay 2016). The timetabling aims at the allocation of events to timetable periods to satisfy a set of hard constraints and minimizing a set of soft constraints (Qu et al. 2009; Mccollum et al. 2010), in which the events are exams, meetings between groups of students and teachers and meetings between classes and teachers. The solution approaches for these timetabling problems include graph coloring method, constraint satisfaction, meta-heuristics, hyper-heuristics and so on.

An important feature of patient scheduling in hemodialysis service to be considered is the multi-function of the resources used in the service. The allocation of the multi-function resources can increase the flexibility of assigning tasks to overcome bottlenecks. In addition, utilizing multi-function resources instead of adding resources during peak demand period can lead to reduced total costs and improved production efficiency. Multi-function resource scheduling is usually witnessed in job/flow shop scheduling problem where the machines possess multiple functions (Pinedo 2012) and various scheduling problem in services industry (Uyar et al. 2013). Multi-skilled employee assignment problem is also similar to multi-function resource scheduling in which researchers usually classified it as scheduling problems of hierarchical skilled employees or non-hierarchical skilled employees (Bruecker et al. 2015).

Patient scheduling in this work differs from past research mainly on optimization objective and characteristics of multi-function resource scheduling. For the optimization objective, the scheduling of patients on hemodialysis aims to a synthetic criteria combined with minimizing the cost of hemodialysis equipment and the number of night shift, taking the life span of the devices into account and especially satisfying patients preferences on time, device and space. In this case, each patient requires to be treated in different number of time slots according to specific treatment plan. In this context, an optimization model to assist patient scheduling is formulated. A basic heuristics and an effective algorithm based on rollout framework are proposed and verified with computational experiments on real cases.

3 Problem description and formulation

Generally, patients in hemodialysis service centers can be classified into three types: conventional, hepatitis B and hepatitis C. For each type of patients, the treatment modality can be high-flux hemodialysis (HD) or hemodiafiltration (HDF). An HDF device can be used for HD treatment. However, it is often not advocated using a device in accordance with downward compatible mode. Therefore, there are six categories of hemodialysis services in total: the conventional HD and HDF, HD and HDF for hepatitis B patients, HD and HDF for hepatitis C patients which are abbreviated as HD, HDF, BHD, BHDF, CHD, CHDF, respectively.

Hemodialysis service centers usually schedule their services in 2-week cycles. There are three shifts (morning, afternoon, and evening shifts) in a workday and each device can provide dialysis treatment to only one patient on a shift. Due to equipment failure, maintenance, repair, reservation or other reasons, the device may not be available in some shifts.

Before treatment, the doctor will give the patient a dialysis plan according to his/her physical condition. For example, one patient with a severe disease needs five treatments including four conventional HD and one conventional HDF in 2 weeks. In general, each patient receives at most one treatment per day. According to medical requirements, the distribution of the number of patients treated in a service cycle should be as uniform as possible.

The task of managers in hemodialysis service center is to arrange treatment time of patients and the corresponding dialysis device according to the functions of devices, the patients preferences (shifts, devices and type of rooms etc.). They usually take the following factors in scheduling.

  1. 1.

    For multi-functional devices, due to the varying cost associated with different functions, higher quality function should be used whenever it is possible. For example, when the device has the function of HD and HDF, it should be given priority to assign an HDF treatment.

  2. 2.

    The evening shifts should be as few as possible.

  3. 3.

    Patients preferences should be satisfied whenever it is possible.

  4. 4.

    Taking the life span of the devices into account, each device should not be overused during a cycle.

Based on the problem description above, some notation is given firstly.

Res is he set of dialysis devices or resources, \(r \in Res\) for scheduling. It is assumed that there are M devices, namely \(Res = \{ r_{1},\ldots ,r_{M}\}\). \(s \in Sk\) is the set of dialysis service categories. Here \(Sk = \{1,\ldots ,6\}\) represent respectively six kinds of dialysis services (HD, HDF, BHD, BHDF, CHD, CHDF).

E is the set of patients. There are N patients, \(E = \{e_{1},\ldots ,e_{N}\}\). \(e \in E\) is the index of patient.

T indicates scheduling period (day), usually \(T = 14\). d is the time serial number, and \(d \in \{1,\ldots ,T\}\).

h denotes shifts, \(h \in \{morning,afternoon,evening\}\) represents shift A, P and N.

p indicates space partition (room) number. There are B partitions. \(p \in \{1,\ldots ,B\}\).

\(k_{rs}\) is device-function coefficient, \(k_{rs} \in \{0,1\}\). \(k_{rs} = 1\) means that device r is able to provide service s, otherwise \(k_{rs} = 0\).

\(a_{rdh}\) is device availability coefficient, \(a_{rdh} \in \{0,1\}\). \(a_{rdh} = 1\) means that device r is available on shift h of day d, otherwise \(a_{rdh} = 0\). From this, the availability matrix of device r, \(\bar{A}_{r}\) can be established:

$$\begin{aligned} \bar{A}_{r} = \left( \begin{array}{ccc} a_{r11} &{}\quad \ldots &{}\quad a_{rT1} \\ \ldots &{}\quad \ldots &{}\quad \ldots \\ a_{r13} &{}\quad \ldots &{}\quad a_{rT3} \end{array} \right) ^T \end{aligned}$$

\(g_{rp}\) is device-partition coefficient, \(g_{rp} \in \{0,1\}\). \(g_{rp} = 1\) shows that device r is located in partition p, otherwise \(g_{rp} = 0\).

\(q_{es}\) is the coefficient of patient treatment prescription and is set as the number of treatments patient e need for each type s.

\(pt_{edh}\) is the coefficient of patient time preference, \(pt_{edh} \in \{0,1\}\). \(pt_{edh} = 1\) if patient e prefers to receive treatment on shift h of day d, otherwise \(pt_{edh} = 0\).

\(pe_{er}\) is the coefficient of patient time preference, \(pe_{er} \in \{0,1\}\). \(pe_{er} = 1\) if patient e prefer to receive treatment using device r, otherwise \(pe_{er} = 0\).

\(pp_{ep}\) is the coefficient of space preference, \(pp_{ep} \in \{0,1\}\). \(pp_{ep} = 1\) if patient e prefer to receive treatment at partition p, otherwise \(pp_{ep} = 0\).

To ensure that patients are treated as uniformly in time as possible (equal number of days between treatments), the candidate treatment patterns (in which days the patient will be treated) can be constructed in advance (as shown in Appendix A). For example, a 2-week treatment pattern (1, 0, 0, 1, 0, 0, 0 , 1, 0, 0, 1, 0, 0, 0) means that the patient is treated on the first, fourth, eighth and eleventh day. The pattern is relatively uniform in time. Let l be the index of treatment patterns and \(\bar{s}_{ld}\) be the treatment pattern coefficient, \(\bar{s}_{ld} = 1\) when the patient is scheduled to be treated with pattern l on day d.

The decision variables in hemodialysis service scheduling are:

$$\begin{aligned} \theta _{el}= & {} \left\{ \begin{array}{ll} 1 &{}\quad patient~e~is~treated~with~pattern~l \\ 0 &{}\quad otherwise \end{array} \right. \\ X_{rdhs}= & {} \left\{ \begin{array}{ll} 1 &{}\quad device~r~provides~sevice~s~on~shift~h~of~day~d \\ 0 &{}\quad otherwise \end{array} \right. \\ Y_{edhs}= & {} \left\{ \begin{array}{ll} 1 &{}\quad patient ~e~ recieves~ service~ s~ on ~shift ~h ~of ~day ~d \\ 0 &{}\quad otherwise \end{array} \right. \\ Z_{edhr}= & {} \left\{ \begin{array}{ll} 1 &{}\quad patient ~e~ is~ treated ~with ~device~ r ~on ~shift ~h ~of ~day~ d \\ 0 &{}\quad otherwise \end{array} \right. \end{aligned}$$

The dialysis service scheduling must satisfy the following constraints:

$$\begin{aligned}&X_{rdhs} \le k_{rs}a_{rdh}, \quad \forall {r,d,h,s} \end{aligned}$$
(1)
$$\begin{aligned}&\sum _{r \in Res}X_{rdhs} \ge \sum _{e \in E}Y_{edhs},\quad \forall {d,h,s} \end{aligned}$$
(2)
$$\begin{aligned}&\sum _{l}\theta _{el} = 1,\quad \forall {e} \end{aligned}$$
(3)
$$\begin{aligned}&\sum _{l}\sum _{d}\bar{s}_{ld}\theta _{el} = \sum _{s \in Sk}q_{es},\quad \forall {e} \end{aligned}$$
(4)
$$\begin{aligned}&\sum _{d}\sum _{h}Y_{edhs} = q_{es},\quad \forall {e,s} \end{aligned}$$
(5)
$$\begin{aligned}&\sum _{l}\bar{s}_{ld}\theta _{el} = \sum _{h}\sum _{s \in Sk}Y_{edhs},\quad \forall {e,d} \end{aligned}$$
(6)
$$\begin{aligned}&\sum _{s \in Sk}X_{rdhs} \le 1,\quad \forall {r,d,h} \end{aligned}$$
(7)
$$\begin{aligned}&\sum _{s \in Sk}Y_{edhs} \le 1,\quad \forall {e,d,h} \end{aligned}$$
(8)
$$\begin{aligned}&\sum _{s \in Sk}X_{rdhs} = \sum _{e \in E}Z_{edhr},\quad \forall {r,d,h} \end{aligned}$$
(9)
$$\begin{aligned}&\sum _{s \in Sk}Y_{edhs} = \sum _{r \in Res}Z_{edhr},\quad \forall {e,d,h} \end{aligned}$$
(10)
$$\begin{aligned}&\theta _{el},X_{rdhs},Y_{edhs},Z_{edhr} \in {\{0,1\}} \end{aligned}$$
(11)

Equation (1) guarantees that device r can be used to provide service s on shift h of day d. Equation (2) denotes that there should be sufficient number of devices to serve the patients. Equation (3) enforces that each patient can choose only one treatment mode and Eq. (4) ensures the number of treatments in the chosen mode equal the requirement for that patient. Equation (5) shows that the requested type of treatment for the patient must be satisfied. Equation (6) uses the treatment mode to ensure each patient only receives one treatment per day and is treated uniformly in time. Equation (7) forces each device to provide only a single type of service on a shift. Equation (8) indicates that each patient can only receive one kind of dialysis treatment on a shift. Equation (9) is that the number of services provided by one device on shift h of day d is equal to the number of people receiving treatment on shift h of day d. Together with Eq. (7), a device can only provide treatment to one patient on a shift. Equation (10) shows the corresponding relation between \(Y_{edhs}\) and \(Z_{edhr}\).

Let \(C_{rs}\) be the cost coefficient for devices providing services. The priority measure is used for the coefficient. For example, a device with conventional HDF function has a higher priority (lower cost coefficient) to be used for conventional HDF than to be used for conventional HD, and cannot be used to treat hepatitis B and C patients. Therefore, its cost coefficient can be expressed as \((2,1,\infty ,\infty ,\infty ,\infty )\) (corresponding to HD, HDF, BHD, BHDF, CHD, CHDF, respectively).

There are usually six criteria in patient scheduling in hemodialysis service:

Criterion 1: To minimize the gross utilization cost of devices.

$$\begin{aligned} CO = \sum _{r}\sum _{d}\sum _{h}\sum _{s}C_{rs}X_{rdhs} \end{aligned}$$
(12)

Criterion 2: To minimize the number of night treatments.

$$\begin{aligned} NS = \sum _{e}\sum _{r}\sum _{d}\sum _{h=3}Z_{edhr} \end{aligned}$$
(13)

Criterion 3–5: To maximize the preference matching (time, device, and space preferences).

$$\begin{aligned} PMT= & {} \sum _{e}\sum _{d}\sum _{h}\left( pt_{edh}\sum _{r}Z_{edhr}\right) \end{aligned}$$
(14)
$$\begin{aligned} PME= & {} \sum _{e}\sum _{r}\left( pe_{er}\sum _{d}\sum _{h}Z_{edhr}\right) \end{aligned}$$
(15)
$$\begin{aligned} PMP= & {} \sum _{e}\sum _{p}\left( pp_{ep}\sum _{d}\sum _{h}\sum _{r}g_{rp}Z_{edhr}\right) \end{aligned}$$
(16)

Criterion 6: To balance the usage among all devices.

$$\begin{aligned} ED = \left\{ \sqrt{ \frac{1}{r}\sum _{r}\left( \sum _{e}\sum _{d}\sum _{h}Z_{edhr}-\overline{\sum _{e}\sum _{d}\sum _{h}Z_{edhr}}\right) ^2}\right\} \end{aligned}$$
(17)

Motivated by real needs, the priorities from Criterion 1 to Criterion 6 are from high to low. Taking this motivation into consideration, a synthetic objective can be constructed in this type of patient scheduling:

$$\begin{aligned} \max ValR = \beta _{1} *\alpha (CO) + \beta _{2} *\alpha (NS) + \beta _{3} *\alpha (PM) + \beta _{4} *\alpha (ED) \end{aligned}$$
(18)

where \(PM = PMT + PME + PMP\) and \(\alpha (*)\) denotes data normalization, which maps the values to [0,1] by using standard min-max normalization. (\(\beta _{1}\), \(\beta _{2}\), \(\beta _{3}\), \(\beta _{4}\) are the weights of CO, NS, PM, ED, \(\beta _{1} \gg \beta _{2} \gg \beta _{3} \gg \beta _{4} > 0\) and \(\sum _{i=1}^{4}\beta _{i} = 1\))

4 Algorithms

University courses timetabling problem (UCTP) has been proven to be NP-hard (Hadidi 2015). If mapping patients to courses, devices to classrooms and assuming that all the devices located in a room, patient scheduling in hemodialysis service can be regarded as a UCTP. Therefore, patient scheduling in hemodialysis service is also a NP-hard problem. In this section, a basic heuristic algorithm and a Rollout based heuristic algorithm are proposed. The main idea of these two algorithms is to schedule the patients one by one. A feasible schedule set is generated for each patient first. Then the algorithm identifies a feasible solution from the set for each patient.

4.1 Definitions related to patients hemodialysis service schedule

In order to facilitate the description of the algorithm, definitions of the patients treatment schedule in three levels are given in the following.

Definition 1

\(\hat{S}_{e}^{1}\) ndicates the first level treatment schedule for a patient, which determines the treatment patterns (on which days a patient will be treated). For example, the first level schedule for a patient being treated on day 1, 6 and 11 is shown in Fig. 1.

Fig. 1
figure 1

The first level treatment schedule for a patient

Full size image
Table 1 The scale of the schedule library in the first level
Full size table
Fig. 2
figure 2

Treatment schedule in the second level for a patient

Full size image
Table 2 The scale of the schedule library in the first level
Full size table

To ensure that patients are treated as uniformly in time as possible (equal number of days between treatments), the feasible number of treatment patterns is small, which is summarized in Table 1 and detailed in Appendix A.

Definition 2

\(\hat{S}_{e}^{2}\) indicates schedule set in the second level for a patient. Besides which day the patient is treated, each schedule of \(\hat{S}_{e}^{2}\) also determines the kinds of service that is received. As shown in Fig. 2, the patient is treated on day 1, 6 and 11 with the conventional HD, HDF and HDF respectively.

Each schedule of \(\hat{S}_{e}^{1}\) can be converted to one of \(\hat{S}_{e}^{2}\) by adding the specific hemodialysis treatment and it can also be done by enumeration method. Since each schedule of \(\hat{S}_{e}^{1}\) corresponds to multiple schedules of \(\hat{S}_{e}^{2}\), the maximum possible size of \(\hat{S}_{e}^{2}\) is shown in Table 2.

Definition 3

\(\hat{S}_{e}^{3}\) indicates schedule set in the third level for a patient. Each schedule of \(\hat{S}_{e}^{3}\) not only determines in which day the patient is treated and which kinds of service, but also determines the device and shift.

Fig. 3
figure 3

Treatment schedule in the third level for a patient

Full size image

As shown in Fig. 3, the patient is treated with conventional HD with device \(r_{1}\) during the morning shift of the first day, with conventional HDF with device \(r_{5}\) during the afternoon shift of the sixth day and with conventional HDF with device \(r_{2}\) during the morning shift of the eleventh day.

According to the definitions above, \(\hat{S}_{e}^{1}\) and \(\hat{S}_{e}^{2}\) are transition schedule libraries, which indicate some constraints of patients schedules. However, \(\hat{S}_{e}^{3}\) is the final schedule library of patient hemodialysis service.

4.2 Basic heuristic algorithm

The basic heuristic algorithm is shown in Fig. 4.

Fig. 4
figure 4

The flow chart of the basic heuristic method

Full size image

The specific algorithm steps are as follows:

  • Step 1: Initialization. Define the set of patients E demand of patients \(q_{es}\), the coefficients of \(k_{rs}\), \(a_{rdh}\), \(pt_{edh}\), \(pe_{er}\), \(pp_{ep}\) and other parameters.

  • Step 2: Based on total number of treatments the patients needed \(\left( \sum _{s \in Sk} q_{es}\right) \), transform the set of patients E to a new set of patients \(E'\) with the order from high to low. If the total number is the same, the order from low to high based on patients IDs.

  • Step 3: Check whether \(E'\) is empty. If \(E' = \emptyset \), the procedure will end. If not, go to the next step.

  • Step 4: Choose the first patient e in \(E'\) and generate a feasible treatment schedule set \(\hat{S}_{e}^{3}\) of e (The details are in Sect. 4.3). Check whether \(\hat{S}_{e}^{3}\) is empty. If \(\hat{S}_{e}^{3} = \emptyset \), the patient fails to be scheduled and add e to the set of patients that are unable to schedule, then go to Step 6. If not, go to the next step.

  • Step 5: Calculate the priority of each schedule in \(\hat{S}_{e}^{3}\). Choose the optimal solution as the final hemodialysis schedule for patient e. Set decision variables \(X_{rdhs}\), \(Y_{edhs}\) and \(Z_{edhr}\) as 1 according to the treatment time, type of dialysis services, shifts and devices.

  • Step 6: Delete patient e from \(E'\). Go to Step 3.

In this basic heuristic, the patient with highest total number of treatments is scheduled first. The reason is that these patients demand more resources and involve complex preferences, and it is difficult to find a feasible schedule for these patients.

The priority value in Step 5 can be calculated according to Eq. (18).

4.3 Generation of feasible schedule set

Fig. 5
figure 5

Generate feasible schedules set

Full size image

In the previous section, the algorithm needs to quickly generate a feasible solution for a patient in Step 4. Since the number of the feasible schedules is very large, in order to improve the efficiency of the algorithm, the feasible schedule based on the patients preference is selected. The flow chart for generating the feasible schedule set is shown in Fig. 5.

Definition 4

\(\bar{R}_{dhs}\) denotes a candidate device that can be used for service s during shift h on day d.

$$\begin{aligned} \bar{R}_{dhs} = \left\{ r|a_{rdh}k_{rs} = 1, \sum _{e \in E}Z_{edhr} = 0, r \in Res\right\} \end{aligned}$$
(19)

Definition 5

\(f_{1}(e,d,h,s)\) quantifies the degree of satisfaction.

$$\begin{aligned} f_{1}(e,d,h,s) = 3w_{1} + 2w_{2} + w_{3} \end{aligned}$$
(20)

\(w_{1}\) denotes the degree of satisfying the time preference, while \(pt_{edh}Y_{edhs} = 1\), \(w_{1} = 1\), otherwise \(w_{1} = 0\). \(w_{2}\) denotes the degree of satisfying the device preference, while \(\sum _{r \in \bar{R}_{dhs}}pe_{er}Z_{edhr} > 0\), \(w_{2} = 1\), otherwise \(w_{2} = 0\). \(w_{3}\) denotes the degree of satisfying the space preference, while \(\sum _{r \in \bar{R}_{dhs}}g_{rp}pp_{ep}Z_{edhr} > 0\), \(w_{3} = 1\), otherwise \(w_{3} = 0\).

The specific algorithm steps are as follows:

  • Step 4.1: Initialize. Obtain \(q_{es}\) and coefficients \(k_{rs}\), \(A_{rdh}\), \(pt_{edh}\), \(pe_{er}\), \(pp_{ep}\). Set \(\hat{S}_{e}^{1} = \emptyset \), \(\hat{S}_{e}^{2} = \emptyset \), \(\hat{S}_{e}^{3} = \emptyset \).

  • Step 4.2: Generate the first level treatment schedule set \(\hat{S}_{e}^{1}\) for the patient via Definition 1\(\left( \sum _{s \in Sk}q_{es}\right) \).

  • Step 4.3: Generate the second level treatment schedule set \(\hat{S}_{e}^{2}\) for the patient via Definition 2 for each schedule in \(\hat{S}_{e}^{1}\) according to the hemodialysis demand (\(q_{es}\), \(s \in Sk\)).

  • Step 4.4: Check whether \(\hat{S}_{e}^{2}\) is empty. If \(\hat{S}_{e}^{2} = \emptyset \), go to Step 4.9; otherwise go to the next step.

  • Step 4.5: Select schedule l randomly from \(\hat{S}_{e}^{2}\), and determine the shift for each service in schedule l. Choose shift A or shift P preferentially when determining the service shift. If no device is available during shift A or shift P, choose shift N; otherwise calculate the value of \(f_{1}(e,d,h,s)\) on shift A and shift P, then choose the one with the higher value.

  • Step 4.6: Check whether all the shifts has been determined. If there is a service whose shift cannot be determined, go to step 4.8; otherwise go to the next step.

  • Step 4.7 Determine the device for each service in schedule l. Add schedule l and the corresponding device into \(\hat{S}_{e}^{3}\) as a new schedule.

First, define the set of candidate device (\(\bar{R}_{dhs}\) assume that patient receive service s on shift h of day d). Then choose the device with the minimum cost coefficient \(C_{rs}\) from \(\bar{R}_{dhs}\). If there are several devices with the same \(C_{rs}\), then choose the device from \(\bar{R}_{dhs}\) by patients device preference and space preference.

  • Step 4.8: Delete schedule l from \(\hat{S}_{e}^{2}\). Then go to step 4.4.

  • Step 4.9: Return \(\hat{S}_{e}^{3}\) to the main algorithm.

Because the generation of \(\hat{S}_{e}^{1}\) and \(\hat{S}_{e}^{2}\) is an enumeration process, and the generation of \(\hat{S}_{e}^{3}\) is based on the patients preference, the schedule set \(\hat{S}_{e}^{3}\) is a subset of all feasible solutions. For Step 4.9, when it is impossible to provide dialysis service for the patient because of the resource limitation, an empty schedule set is returned.

4.4 Rollout algorithm

In Sect. 4.2, feasible schedule is selected by the priority rules from the feasible schedule library. In order to improve the quality of the solution, a rollout algorithm is presented to solve the hemodialysis service scheduling problem.

4.4.1 A brief introduction of rollout algorithm

Rollout algorithm has been proposed to solve discrete optimization problems by Bertsekas et al. (1997). It usually embeds a basic approach to the Rollout iterative process. It uses a basic strategy as the starting point, and gets a new strategy through rollout to approach optimal strategy step by step based on the new strategy. Rollout algorithm converges quickly, and it can obtain the preferable solution on the basis of the full inheritance of the basic solution algorithm (Bertazzi 2012). Based on these advantages, Rollout algorithm has successfully been used to tackle various optimization problems (Guerriero 2008; Guerriero et al. 2002; Guerriero and Mancini 2005; Xu et al. 2008; Guerriero et al. 2015).

4.4.2 Rollout algorithm for patient scheduling in hemodialysis service

Fig. 6
figure 6

The flow chart of the rollout algorithm

Full size image

In the Rollout algorithm, the cost-to-go function is constructed to evaluate each schedule in the feasible schedule library to determine the final schedule of hemodialysis service. With a single step forward iterative computation, the algorithm has the ability to make decisions in advance, which achieves the purpose of improving the quality of the solution from the basic heuristic approach. The rollout algorithm framework is shown in Fig. 6.

As shown in Fig. 6, the Rollout algorithm is consistent with the basic heuristic approach in Sect. 4.2. The calculation of the cost-to-go function is through determining the hemodialysis service schedule of the patient that has not been scheduled (other patients have been scheduled through the basic heuristic) to build a complete hemodialysis service schedule for all patients.

In this process, there are two key points:

  1. 1.

    The cost-to-go function is used to evaluate the rationality of a patient to be treated by a feasible schedule. Therefore, it is assumed that the patient will eventually adopt the plan.

  2. 2.

    The Cost-to-go function is used for schedule evaluation, not for the final schedule selection. Therefore, the values of \(X_{rdhs}\), \(Y_{edhs}\) and \(Z_{edhr}\) are not changed (they are replaced by \(X'_{rdhs}\), \(Y'_{edhs}\) and \(Z'_{edhr}\) in this step).

For the generation of the complete scheduling plan, the computational rules of the cost-to-go function values are also defined as Eq. (18).

Further analysis of the Rollout algorithm shows that, the algorithm calculate cost-to-go function value for each feasible schedule. The size of the feasible solution can reach 140 in Sect. 4.2. Its efficiency will be very low. Therefore, the scale of \(\hat{S}_{e}^{3}\) will be set in a certain size SC (\(SC \in [1,140]\)), then the optimal schedule is selected by using cost-to-go. The algorithm is named as Rollout-SC. Obviously, the effect of Rollout-1 and the basic heuristic algorithm is the same, and Rollout-140 do not limit the size of \(\hat{S}_{e}^{3}\).

5 Numerical experiments

This section is to evaluate the performances of the proposed algorithms. Firstly, a new definition is given as follows:

PMS: treatment times that meet all of a patient preference on time, device and partition.

The algorithms are coded in Java programming language, JDK 1.7 and run in Windows 7, CPU 3.20GHz, 4.00GB memory.

5.1 Test cases

In order to test and compare the performance of the algorithms, practical cases are collected from Hemodialysis Service Center P in Wuhan, China and extend them to adapt to various characteristics of the problem.

5.1.1 Data source

Actual dialysis schedules are selected from the hemodialysis service center in two periods, 12.31 2014 to 1.13 2015, and 3.16 2015 to 3.29 2015, as basic data for computational experiments. The first period is a peak of hemodialysis treatment, when 174 patients were scheduled for treatments, and the second period is common, of which 118 patients were scheduled. There are 5 space partitions in the center, \(P = 5\). The number of device is given in Table 3.

Table 3 The number of devices in hemodialysis service center
Full size table

It is assumed that there is no failure or other reason which causes the unavailability of devices. According to the timetable of the hemodialysis service center, nights and Sunday afternoon are not work time expect Wednesday. Therefore, the availability matrix of every device is shown in Formula (21).

$$\begin{aligned} \bar{A}_{r} = \left( \begin{array}{@{}cccccccccccccc@{}} 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 \\ 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 0 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 1 &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 1 &{}\quad 0 &{}\quad 0 &{}\quad 0 &{}\quad 0 &{}\quad 0 &{}\quad 0 &{}\quad 1 &{}\quad 0 &{}\quad 0 &{}\quad 0 &{}\quad 0 \\ \end{array} \right) ^{T} \end{aligned}$$
(21)

5.1.2 Adjustment of case scale

Taking into account the case on 2015.3.16 to 2015.3.29 is a peak of dialysis treatment, the experiments are conducted on the basis of the periodic data. The problem solving ability of algorithm on different device scale and patient scale should be verified in the experiments, therefore two strategies are utilized to adjust case scale as follows:

  1. 1.

    Add or remove some conventional HD devices or conventional HDF devices randomly to adjust device scale based on the data from 2015.3.16 to 2015.3.29. This strategy takes into account the maximum number of the conventional HD or HDF devices shown in Table 3, which is suitable for a certain decrease on device numbers.

  2. 2.

    Remove some conventional patients randomly to reduce the patient scale based on the data from 2015.3.16 to 2015.3.29, and select some conventional patients from 2014.12.31–2015.1.13 to join the period from 2015.3.16 to 2015.3.29 to increase the patient scale. The strategy takes into account the maximum number of conventional patients and the maximum number of the conventional HD devices.

5.1.3 Adjustment of patients preferences

In reality, hospital manager usually arranges patients manually without considering patient preferences. To reflect the actual requirements, the patients preferences on time, device and partition will be replenished according to patient treatment plan. An example treatment schedule \(e_{1}\) is shown as Table 4:

Table 4 Treatment schedule of \(e_{1}\)
Full size table

(1) Add preferences on time

Without loss of generality, time preference of a patient can be either shift A or shift P. In the generation of time preference, add all the shift A preference if the majority of the patients in the treatment is in the morning, otherwise add P class preference.

(2) Add preferences on device

Scheduling system sets that a patient can choose 5 kinds of device preferences. Set preferences on device according to the data obtained from the patient’s treatment device. When the number of treatment device is more than 5, then choose 5 devices from this.

(3) Add preferences on partition

The preferences on partition are the preferences of treatment room and this preference constraint is loose. Thus, there is only one preference for each patient. It will treat patient with the first treatment room as the patient’s preference for the partition.

5.2 Parameter setting of SC in the RA

In this section, rule ValR can be used to limit patients size to SC, which can improve the computation speed. In order to analyze the possible influence of SC on the algorithm, the data of the 2014.12.31–2015.1.13 is used to conduct experiments under different SCs, and the results are shown in Fig. 7 (\(\beta _{1} = 10 *\beta _{2} = 100 *\beta _{3} = 1000 *\beta _{4} = 1000/1111\)).

Fig. 7
figure 7

Experimental results when SC changes. a The trend of PM and ED. b The trend of ValR. c The trend of runtime

Full size image

Figure 7a shows that: when \(SC \le 10\), the increase of SC increases the number of preference satisfaction (PM) and the use deviation ratio of device (ED) significantly; when \(10 < SC \le 40\), both PM and ED have a fluctuation process, especially for ED; when \(SC > 40\), PM and ED tend to be stable. Figure 7b shows that: when \(SC \le 10\), the total target value (ValR) increases with the increasing of SC because of the influence of PM and ED; when \(10 < SC \le 30\), the ValR slow growth; when \(SC > 30\), ValR tends to be stable. The possible reason of ValR reaches its maximum when \(SC = 40\), is that the preliminary plans of Rollout algorithm is randomly generated, therefore, the best solution is not guaranteed. Figure 7c indicates that the increase of SC leads to a fast increasing on the algorithm solution speed, which then slows down. (All patients are scheduled successfully in this experiment, and \(CO = NS = 0\)).

Hence it can be concluded that: 1) To satisfy the patient’s preferences and the balanced use of device is a pair of conflicting goals; 2) Compared with the basic heuristic approach (Rollout-1), Rollout can improve the quality of the solution, and increasing the value of SC can improve the probability of obtaining a better solution of, not necessarily to improve the quality of the solution; 3) Considering the effect of the algorithm and solving the problem in 10min, it will be set \(SC = 40\) in the following experiments.

Table 5 Patient need from 2015.3.16 to 2015.3.29
Full size table

5.3 A case study: compared with manual scheduling

In this section, the results of the algorithms are compared with manual scheduling for the same case. There is no general way to arrange patients manually which is based on management’s experience and habits. So the data of manual scheduling is obtained from the real data of cases from 2015.3.16 to 2015.3.29 without adjusting and considering patients preference. The patient needs are shown as Table 5.

Without considering the patient preference, the preference satisfied percent is 100%. Set \(\beta _{1} = 10 *\beta _{2} = 100 *\beta _{3} = 1000 *\beta _{4} = 1000/1111\) (the setting of \(\beta \) will be analyzed in details in Sect. 5.5.2). The experiment results are shown in Table 6. All patients can be successfully scheduled using the primary function of devices in the results of three kinds of scheduling plan. So, \(CO = 0\).

Table 6 Results for various scheduling methods
Full size table

The second and fourth column of Table 6 shows that: when not considering the patient preference, the uniformity of patient treatment time using Rollout algorithm is better compared to manual scheduling; Rollout algorithm makes balanced device use; and Rollout algorithm has a better ValR.

When using the RA solving speed is significantly slower than that using the basic heuristic approach. However, the result calculated by Rollout approach has not arrangement for evening shift, heuristic approach is scheduled for the evening, and the former balance device is slightly better than basic heuristic approach.

5.4 Multi-case study: compared with heuristic approach

Comparative computational experiments are conducted on different scales with taking into account the patients preference in this section, and Rollout-40 and heuristic approach are compared and analyzed. The results are shown as Table 7 (\(\beta _{1} = 10 *\beta _{2} = 100 *\beta _{3} = 1000 *\beta _{4} = 1000/1111\)).

Table 7 Results on different scales taking into account the patient preference
Full size table

Through the comparison of results, it can be found that Rollout-40 performs better than heuristic approach in the time, device and partition preference satisfaction of scheduling. Nevertheless, variance of device use time and ValR for the former are greater than the latter, and computing time is greatly lengthened. All cases in Table 7 satisfy this law.

In some cases algorithm begins to schedule night shift, even use HDF device as HD (that is to say \(CO \ne 0\)) when there are too many patients and lacking of corresponding device, such as case \(M = 19, N = 100\). However, as the case \(M = 28, N = 150\) shows, the RA results at taking fewer night shifts.

In conclusion, Rollout performs better in satisfying patients preference and arranging fewer night shifts compared to heuristic approach in these cases.

Fig. 8
figure 8

Comparison of the RA-40 and the Heuristics. a The trend of PM, PMT, PME, PMP and ED. b the trend of ValR

Full size image

To further analyze the influence when using two algorithm on different scales, the experiment is conducted in the situation where \(M =34\) and patient number is different, the result is shown as Fig. 8. The vertical ordinate in Fig. 8 represents percentage of that the results using Rollout-40 compared with heuristic approach, and horizontal ordinate is the number of patients, shown as follows:

$$\begin{aligned} Percent(index) = \frac{Rollout(index)-Heuristics(index)}{Heuristics(index)} *100\% \end{aligned}$$
(22)

Of which, index are PM, ED etc. Rollout(index) is the result using the RA about index; Heuristics(index) is the result using heuristic approach about index.

Figure 8a shows that number PM of treatment satisfying patient preference increases from 1.38 to 2.22% when using the RA compared with using heuristic approach, and the patient scale does not affect the improvement. In Fig. 8, PME curve is above PM, whereas PMP curve is always below the PM and even negative, and PMT curve is close to PM. It indicates that the scheduling schedule generated by Rollout method performs better in satisfying patient preference compared to heuristic approach, and the improvement is mainly to improve the satisfaction of patients’ device preferences.

On the other hand, ED curve shows that the device using variance generated by the Rollout algorithm is more than 5% of the basic heuristics, it can reach as high as 20.15%. Rollout focuses more on satisfying the patients preference, while the patients preference, in particular, the device preference often breaks the balanced use of device. With the increase of the number of patients, the ED curve has a high trend, which indicates that with more and more patients with preference joining, the imbalance in the use of device will gradually appear.

Figure 8b shows that that the scheduling results obtained by the Rollout algorithm can increase the total target value (ValR) compared with the basic heuristic.

In short, the Rollout algorithm has a stable improvement comparing with the basic heuristic approach on satisfying the patients preferences.

5.5 Sensitivity analysis of parameters in the problem

The patient scheduling in hemodialysis service in which validity and rationality of scheduling results are not only related to the quality of scheduling algorithm, but also limited by the parameters of the problem such as scale of devices, coefficients of each criterion.

5.5.1 Scales of devices

As a common sense, the scale of hemodialysis devices in the problem can make obvious impact on the scheduling results. With a settled patient scale (the data of cases from 2015.3.16 to 2015.3.29), the Rollout-40 algorithm is used to conduct the computational experiment with different scale of conventional HD devices or conventional HDF devices. The results are shown in the Tables 8 and 9 ( \(\beta _{1} = 10 *\beta _{2} = 100 *\beta _{3} = 1000 *\beta _{4} = 1000/1111\)).

Table 8 Results on different scales of conventional HD devices
Full size table

Case \(M=23\) in Table 8 is the original case from 2015.3.16 to 2015.3.29. With the comparison of results, it can be concluded that: with the increase in scale of conventional HD devices, the number of treatment times that meet the preference of time, device and partition increased, there is some similar conclusion on the total target value (ValR). The runtime also has a gradual tendency to increase.

Table 9 Results on different scales of conventional HDF devices
Full size table

Case \(M=5\) in Table 9 is the original case from 2015.3.16 to 2015.3.29. Due to the small number of HDF devices, it can be concluded that the experiments using the Rollout algorithm to ensure the accuracy of the conclusion. With the comparison of results, the conclusions are roughly similar to those of the Table 8, the only difference is that when the scales of conventional HDF is greater than 6 pieces, the ED will increase because of the demand of HDF is relatively small and ValR also decrease. On the other hand, it is obvious that Rollout-40 focuses more on satisfying the patient’s preference, while the patients preference often breaks the balanced use of device.

In conclusion, relatively small devices scale will make scheduling results unsatisfactory, and an excessive one will increase the amount of calculation, resulting in increased computation time.

Table 10 Results of different \(\beta \) values
Full size table

5.5.2 Parameters \(\beta _{1}\) to \(\beta _{4}\)

RA-40 is also taken in this experiment. According to the description of \(\beta \) values in Sect. 4.2 (\(\beta _{1} \gg \beta _{2} \gg \beta _{3} \gg \beta _{4} > 0\) and \(\sum _{i=1}^{4}\beta _{i} = 1\)), consider of the convenience for experimental manipulation, it is assumed that \(\beta _{1} = k *\beta _{2} = k *k *\beta _{3} = k *k *k *\beta _{4} \) and change the value of k, instead of changing the values of each \(\beta \) to analyze the influence of different \(\beta \) values on the experimental results. Part of the data of the 2014.12.31–2015.1.13 (19 devices and 100 patients, by the strategies in Sect. 5.1.2) is used in these experiments, and the results are shown in Table 10.

Table 10 shows that: as long as \(k \ge 4\) is established, no matter how large the value of k is, it does not affect the final scheduling scheme and the runtime. When \(k \le 10\), the value of ValR increases with the increase of k. When \(k > 10\), ValR decreases with the increase of k.

In conclusion, too small k cannot reasonably reflect the precedence relationship among the different objectives. However, a too large one can keenly lead to different impacts of various criteria on the synthetic objective to be revealed.

6 Conclusion

A patient scheduling problem faced by hemodialysis service centers in China is studied in this paper. This problem considers a synthetic objective combined with the gross utilization cost of devices, the number of night treatments, patients preferences on time, device, space, and equilibrium of the devices. A 0–1 integer programming model is formulated for this problem and the basic heuristic method is developed in which three levels of treatment schedules set are constructed one by one. Based on these, a Rollout framework for solving the problem is presented and the algorithm selects schedules through the evaluation of the cost-to-go function to improve the quality of solution. Finally, with real cases of a hemodialysis service center in Wuhan China, comparison tests using the rollout algorithm are conducted to the manual scheduling, and the basic heuristic method. Compared with the basic heuristic method, the Rollout algorithm can effectively improve patients degree of satisfaction and reduce the number of night shifts and HDF devices used to do the HD. Sensitivity analyses on scales of devices and coefficients of various criteria are carried out to reach some interesting conclusion.

In this paper, the patient scheduling problem is formulated as a model with a synthetic objective combined with 6 criteria. The coefficients of these criteria are set by managers according to their experience in this work. It is more reasonable if some multi-criteria decision making methods such as AHP, ANP etc. are taken in setting these coefficients. It will be a meaningful work in practice in the future. Developing meta-heuristics or hyper-heuristics for solving the synthetic objective problem is another direction. Additionally, the multiple criteria can be described as multiple objectives and then the problem can be formulated as multi-objective or many-objective programming models. Designing multi-objective or many-objective evolutionary algorithm is worthy of paying more attention.