Stochastic Resource Allocation with Time Windows

Abstract

The stochastic resource allocation problem with time windows (SRAPTW) refers to a class of combinatorial optimization problems which are aimed at finding the optimal scheme of assigning resources to given tasks within their time windows. In SRAPTW, the capability of resources to accomplish tasks is quantitatively characterized by probability. The expected allocation scheme should include not only the task-resource pairings but also their allocation time. This paper formulates SRAPTW as a nonlinear mixed 0–1 programming problem with the objective of maximizing the reward of completing specified tasks. Then, a general encoding/decoding method is proposed for the representation of solutions, and several different problem-solving methodologies are presented and compared. Results of computational experiments show that the utilization of SRAPTW-specific knowledge can bring in excellent performance, and a constructive heuristic combining maximal marginal return strategy and maximal probability strategy has remarkable advantages, especially in larger-scale cases.

1 Introduction

The stochastic resource allocation problem with time windows (SRAPTW) refers to a class of combinatorial optimization problems which are aimed at finding the optimal scheme of assigning resources to given tasks within certain time windows. As a complex variant of the stochastic resource allocation problems (SRAP) [1, 2], a prominent feature of SRAPTW is that the capability of resources to accomplish tasks is quantitatively characterized by probability within certain time windows.

SRAPTW widely exists in the control of complex systems which involve the allocation of multiple sensors and actuators for various control tasks. When sensors or actuators are used for moving targets especially those time-critical ones, the time windows of resource allocation are one of the key factors to check the feasibility of task scheduling and execution.

Previous studies on SRAP, such as sensor-target assignment problem and weapon-target assignment problem, were focused on the allocation of resources to tasks without time windows [1,2,3,4,5,6,7,8]. In fact, time windows widely exist in the resource allocation problems from diverse fields, e.g., emergency and rescue [9], communication channel management [10], traffic resources management [11], transportation and logistics [12,13,14], defense resources allocation [15]. Unlike common resource allocation problems involving time windows, the success probability of resources to accomplish tasks is time-dependent in SRAPTW. Therefore, the time windows in SRAPTW not only bring constraints, but also relate to the distribution of success probability, which means the task performance of resources is sensitive to the execution time of tasks. In this sense, solving SRAPTW involves the matching of resources and tasks as well as determining the execution time of tasks, so as to optimize the overall task performance.

This paper is aimed at formulating SRAPTW and proposing efficient strategies and algorithms for solving it. The contribution of the paper is summarized as follows:

• 1) Formulate the SRAPTW as a nonlinear mixed 0–1 programming problem.

• 2) Provide a general encoding/decoding method for representing solutions and designing search algorithms to solve SRAPTW.

• 3) Propose four different SRAPTW algorithms which combine knowledge-dependent constructive heuristic and random search. Conduct comparative computational experiments to identify their pros and cons.

The rest of the paper is structured as follows. Section 2 presents the problem formulation. Section 3 presents the proposed algorithms. Section 4 presents computational experiments. Section 5 concludes the paper.

2 Problem Formulation

2.1 Problem Description

Stochastic resource allocation (SRA) refers to the scenarios in which the capability of a resource (e.g., a person or a robot) to complete tasks is characterized by probabilities. An allocation scenario of m resources and n tasks is considered. The probability of the ith resource ($i=\mathrm{1,2},\cdots ,$ m) to complete the jth task ($j=\mathrm{1,2},\cdots ,$ n) also depends on the allocation time. So, if the probability, denoted by Pij(t), is lower than certain threshold or even becomes 0, the allocation will be abandoned. Generally, the allocation of resources to tasks needs to be handled within specified time windows. Thus, the problem is an SRA problem with time windows (SRAPTW). Assume that each task, once accomplished, will bring a reward. Denote the reward of the jth task by v_j. The objective of SRAPTW is to maximize the total reward of allocating resources to tasks within specified time windows. Besides, resources when assigned usually have to satisfy some constraints. For example, for one resource, there will be some time interval between one allocation and its next, as well as an allowable maximal number of tasks to be allocated. For each task, it is allowed that multiple resources can be used but each resource can be used only once.

2.2 Optimization Model

The SRAPTW can be formulated as an optimization problem. Assume that the execution of tasks about each resource is mutually independent. Then, the objective of the problem can be presented as follows:

$$\mathrm{max }\,{J}_{1}(X,T)=\sum\nolimits_{j=1}^{n}{v}_{j}\left(1-\prod\nolimits_{i=1}^{m}\left(1-{p}_{ij}({t}_{ij}){x}_{ij}\right)\right)$$

(1)

where ${J}_{1}(X,T)$ represents the objective function, X and T are the decision matrices, $X={\left[{x}_{ij}\right]}_{m\times n}$ represents the resource-task allocation relation, $T={\left[{t}_{ij}\right]}_{m\times n}$ represents the task execution time for each resource-task pair when allocated, and $P={\left[{p}_{ij}({t}_{ij})\right]}_{m\times n}$ represents the success probability that a resource can accomplish a task at specific time. A detailed description of other parameters is presented in Table 1.

The objective shown in Eq. (1) can be equivalently converted to the following

$$\mathrm{min }{J}_{2}(X,T)=\frac{\sum_{j=1}^{n}{v}_{j}\prod_{i=1}^{m}\left(1-{p}_{ij}({t}_{ij}){x}_{ij}\right)}{\sum_{j=1}^{n}{v}_{j}}$$

(2)

where ${J}_{2}(X,T)$ is a normalized objective function.

Table 1. Explanation of Model Parameters.

The following constraints have to be satisfied.

Range of Decision Variables.

It is obvious that the allocation variable is of 0–1 type and the allocation time should be determined within specified time window.

$${x}_{ij}\in \{\mathrm{0,1}\}, \forall i\in \{\mathrm{1,2},\cdots ,m\}, \forall j\in \{\mathrm{1,2},\cdots ,n\}$$

(3)

$${t}_{ij}\in [{t}_{ij}^{-},{t}_{ij}^{+}], \forall i\in \{\mathrm{1,2},\cdots ,m\}, \forall j\in \{\mathrm{1,2},\cdots ,n\}$$

(4)

Maximal Number of Tasks to be Allocated for Each Resource.

For resource i, the maximal number of tasks which can be allocated to it should not exceed n_i.

$$\sum\nolimits_{j=1}^{n}{x}_{ij}\le {n}_{i}, \forall i\in \{\mathrm{1,2},\cdots ,m\}$$

(5)

Maximal Number of Resources to be Allocated for Each Task.

For task j, the maximal number of resources which can be allocated to it should not exceed m_j.

$$\sum\nolimits_{i=1}^{m}{x}_{ij}\le {m}_{j}, \forall j\in \{\mathrm{1,2},\cdots ,n\}$$

(6)

Time Interval for Each Resource to Execute Two Tasks.

For resource i, the execution of any two tasks assigned to it should be separated at least ${{{\tau}}}_{i}$.

$${|t}_{ij}-{t}_{ik}|\ge {\tau }_{i}, \forall i\in \{\mathrm{1,2},\cdots ,m\}, \forall j,k\in \{\mathrm{1,2},\cdots ,n\}, j\ne k$$

(7)

To sum up, the optimization model for SRAPTW can be formulated as follow:

$$\mathrm{min }\,{J}_{2}\left(X,T\right),\mathrm{ s}.\mathrm{t}. \left(3\right)\sim \left(7\right).$$

Obviously, SRAPTW is a nonlinear mixed-variable programming problem.

2.3 Problem Analysis

From the problem formulation shown above, the following properties can be derived:

Property 1:

The more resources are allocated without violating any constraints, the better the objective value will be. Stated another way, for any X, if more x_ij can become 1, J₂(X,T) can be improved.

Property 2:

For any X, the larger ${p}_{ij}({t}_{ij})$ is, the better the objective value.

Property 3:

If all ${t}_{ij}^{*}=\mathrm{argmin}({p}_{ij}({t}_{ij}))$,$\forall i\in \{\mathrm{1,2},\cdots ,m\}, \forall$ j $\in \{\mathrm{1,2},\cdots ,n\}$, do not have conflicts w.r.t. constraints (7), then ${{t}_{ij}=t}_{ij}^{*}$ will be a necessary condition for any ${x}_{ij}=1$.

As demonstrated later, these properties are beneficial to design efficient problem-specific strategies or operators to solve SRAPTW.

3 Algorithm Design

3.1 Solution Representation

The decision variables X (binary valued) and T (real valued) are straightforward representation of solutions to SRAPTW. However, it will be more convenient to utilize the problem-domain knowledge in SRAPTW by use of permutation-based encoding and decoding schemes.

According to Property 1, for a feasible X which does not violate the constraints (3), (5) or (6), changing more ${x}_{ij}$ from 0 to 1 will bring the improvement of the objective value. In this sense, for an empty X (all zero elements), the order of making each ${x}_{ij}$ become one while ensuing constraint satisfaction can be a key factor to generate a feasible and even high-quality solution. In other words, the permutation of all resource-task pairs can be used as a main component of solution representation.

For the execution time regarding each pair, we can determine the value of each t_ij from its time window according to certain rules, e.g., unbiased random sampling or biased heuristic selection.

To sum up, the permutation of all resource-task pairs and corresponding execution time can be used to represent a solution to SRAPTW. The detailed encoding-decoding scheme is described as follows.

Encoding Scheme. For a resource-task pair (briefly called an RT pair) denoted by i-j ($i\in \{\mathrm{1,2},\cdots ,m\}$,$j\in \{\mathrm{1,2},\cdots ,n\}$), we use the number n $\times$(i−1) + j to represent it. Then, the permutation of all RT pairs is formally a permutation of the integers from 1 to m $\times$ n. The matrix of the execution time regarding each RT pair, together with the permutation, constitutes the encoding scheme.

For example, for an SRAPTW of 2 resources and 2 tasks with time windows TW₁₁ = [0, 0.1], TW₁₂ = [0.3, 0.4], TW₂₁ = [0.2, 0.3] and TW₂₂ = [0.5, 0.9], the permutation 4-1-3-2 and the matrix of execution time [0.05, 0.38; 0.25, 0.78] represent a solution.

Decoding Scheme. For a given permutation Pe and a matrix of execution time$T={\left[{t}_{ij}^{\prime}\right]}_{m\times n}$, ${x}_{ij}=1$ for each RT pair will be checked, in the order indicated by Pe, to see if it violates constraints (5), (6) or (7). If no constraints violation occurs, then let ${x}_{ij}=1$ and ${t}_{ij}={t}_{ij}^{\prime}$; otherwise, let ${x}_{ij}=0$ and ${t}_{ij}=\infty$.

Based on the above representation scheme, a feasible solution can be generated by decoding after determining a permutation of all RT pairs and a execution time matrix. Both the permutation and the execution time matrix can be generated in different ways. For example, they can be generated by random sampling or constructive heuristics.

3.2 Constructive Heuristics

Constructive heuristics represent a philosophy of straightforwardly generating a solution to a complex problem by determining its components step by step, rather than implementing samplings in solution space as widely adopted in various search algorithms such as improvement heuristics and metaheuristics [16,17,18]. In comparison with search algorithms, constructive heuristics do not need function evaluations to improve solutions in an iterative way. To design efficient constructive heuristics, the permutation of all RT pairs and the execution time matrix can be generated by utilizing SRAPTW-specific knowledge, e.g., the three properties shown in Subsect. 2.3. To utilize Properties 2 and 3, the execution time will be set to ${t}_{ij}^{*}$ if it does not violate any constraint. The permutation can be constructed by using the rule-based method based on maximal marginal return (MMR) [6, 19]. The marginal return means the improvement of the objective value with reference to its current value brought by choosing one RT pair. MMR is a rule for constructing a solution incrementally, and in each step, the RT pair with maximal marginal return without constraint violation will be added into the allocation scheme. In fact, since the MMR-based procedure can generate a complete solution directly, it is unnecessary to get the permutation as a preliminary which is de facto implicated in the procedure. To save space, the rationale of MMR will not be presented here in more detail, and interested readers may refer to the reference [6]. Instead, we provide the pseudo-code of the procedure in Procedure 1.

Since the core of this constructive heuristic is the MMR-based procedure combined with the execution time setting based on maximal probability, we name it by MMR-MP. The time complexity of determining all the execution time with maximal probability is O(mnl) where l represents the number of time samplings in each time window. From the pseudo-code of MMR-MP, the worst-case time complexity of MMR procedure is O(m²n²). So, the time complexity of MMR-MP is O(m²n² + mnl).

3.3 Random Search

Different from the constructive heuristic MMR-MP, the random search here refers to certain unbiased search process to find better solutions. Since any solution involves two parts, i.e., the permutation and the execution time matrix, we have three different strategies to implement random search:

Strategy 1: MMR for permutation (implicit) & Random sampling of execution time

Strategy 2: Random permutation & Execution time with maximal probability

Strategy 3: Random permutation & Random sampling of execution time

Random permutation can be achieved in MATLAB by the command randperm(n $*$ m) while the random sampling of execution time can be conducted by implementing ${{t}_{ij}=t}_{ij}^{-}+({t}_{ij}^{+}-{t}_{ij}^{-})\times rand$ where rand is a random number following the uniform distribution in (0,1). The three versions of random search resulting from these strategies are named MMR-RS, RP-MP and RP-RS, respectively. In fact, similar to MMR-MP, both MMR-RS and RP-MP utilize SRAPTW-specific knowledge while maintaining certain randomness. In contrast, RP-RS is a pure random search without search bias. Denote the allowable number of random samplings (function evaluations) by N. Then, the time complexity of MMR-RS, RP-MP and RP-RS is O(Nm²n²), O((N + l)mn) and O(Nmn), respectively.

4 Computational Experiments

The goal of the computational experiments is to validate the performance of the proposed algorithms including MMR-MP, MMR-RS, RP-MP and RP-RS. All tests are conducted on a Lenovo Laptop X1 with Intel Core i7 CPU (1.8GHz) and 16GB RAM. For each SRAPTW instance, the random search algorithms including MMR-RS, RP-MP and RP-RS will run 25 times for making a statistical analysis of results.

4.1 Experiments Configuration

Test case generator: The task rewards are generated by following a uniform distribution ${v}_{j} \sim U$(1,1 + $\delta$) where U(a,b) represents a uniform distribution in the interval (a,b), and $\delta$ is a control parameter. All time windows ${TW}_{ij}$ are randomly generated from the interval (0,1). The success probability ${p}_{ij}\left({t}_{ij}\right) \sim N$(${\mu }_{ij}$,${\sigma }_{ij}^{2}$) where $N(\mu ,{\sigma }^{2})$ represents a Gauss distribution with mean $\mu$ and standard deviation $\sigma$. Let ${\mu }_{ij} \sim U$(0,1) and ${(\sigma }_{ij}-\upvarepsilon ) \sim U$(0,${t}_{ij}^{+}-{t}_{ij}^{-}$) where $\upvarepsilon$ is a positive number for regulating the distribution of success probability. A larger $\upvarepsilon$ implies a smaller change of success probability within corresponding time window. For constraints (5) and (6), ${m}_{i}=\lceil\alpha \times {rand}_{i}\rceil$, ${n}_{j}=\lceil\beta \times {rand}_{j}\rceil$ where $\lceil\cdot \rceil$ is the ceiling operator, and $\mathrm{\alpha }$ and $\upbeta$ are control parameters, and ${rand}_{i}$, ${rand}_{j} \sim$ U(0,1). For constraint (7), ${\tau }_{i} \sim U({\tau }_{i}^{-},{\tau }_{i}^{+})$ where ${\tau }_{i}^{-}$ and ${\tau }_{i}^{+}$ are control parameters.

The following cases are generated.

In each case, ten different instances are generated by following the probability distribution preset in the test case generator. Detailed data about the total 100 instances and related results will be released online and can be acquired by contacting the authors.

Parameter Setting: For all random search algorithms, the maximal number of function evaluations (samplings) is set as $10\times m\times n$. For determining the time for the maximal probability in each time window, even samplings within [0,1] are conducted and the minimal spacing is set to 0.001.

4.2 Performance Comparison

For each instance in each case, the results of all algorithms, i.e., obtained objective values in 25 runs, will be compared by Wilcoxon ranksum test with 0.05 significance level. For two algorithms A and B, the indicator of the ranksum test h = 1 means the performance of A is significantly different from that of B. In this case, if the mean of the results obtained by A is smaller (larger) than that by B, then A gets a score of + 1 (−1) and B gets a score of −1 (+1); otherwise, both A and B get zero. In each case, the highest and lowest scores are 30 and –30, respectively. The scores of four algorithms in 10 cases (totally 100 instances) are summarized in Table 2. To save space, detailed results for each instance will not be presented. The average runtime of the four algorithms in different cases is shown in Fig. 1.

Table 2. Statistical results of algorithm comparison (scores)

Fig. 1.

The average runtime of four algorithms in different cases.

From Table 2, the following results can be found:

• 1) RP-MP has obvious advantages in Cases 1, 2, 3 and 4; in contrast, it is not a good solver in Cases 6 and 10;

• 2) MMR-MP takes the first place in Cases 7, 8, 9 and 10 (especially wins all in the larger-scale Case 10). However, it was defeated by RP-MP in Cases 1, 2, 3, 4 and 5 and performs poorly in Cases 1 and 2.

• 3) MMR-RS gets the highest score in Case 6, and shows its advantage over RP-MP and RP-RS but loses to MMR-MP in Cases 9 and 10.

• 4) RP-RS has no advantage in almost all cases. In Cases 7, 8, 9 and 10, it was completely defeated by the other algorithms. This implies that pure random search without use of problem-specific knowledge is inefficient, though it can theoretically cover the solution space and find the optimal solution with sufficient samplings.

• 5) It seems that the MMR strategy generally has dominant advantages in larger-scale cases, which is supported by (a) the superiority of MMR-MP over RP-MP in Cases 6 to 10, (b) the superiority of RP-MP over MMR-MP in Cases 1 to 5, and (c) the superiority of MMR-RS over RP-RS. The comparison implies that the permutation of RT pairs will play a more important role in determining the quality SRAPTW solutions as the scale of SRAPTW increases.

• 6) It seems that the MP strategy has advantages in smaller-scale cases, which is supported by (a) the superiority of MMR-MP over MMR-RS in Cases 7 to 10, (b) the superiority of MMR-RS over MMR-MP in Cases 1, 2 and 5, and (c) the superiority of RP-MP over RP-RS. Obviously, MP plays a crucial role in generating high-quality solutions for smaller-scale cases, and it is also a necessary strategy for guaranteeing the performance of MMR-MP.

The average time cost of the four algorithms shown in Fig. 1 is consistent with the analysis about computational complexity in Sect. 3. Obviously, MMR-MP has the lowest time cost in all cases. The time costs of RP-RS and RP-MP are very close, which implies that the cost of obtaining the maximal probabilities (regulated by the parameter l) is negligible as compared to that of determining the permutation of RT pairs (regulated by the number of function evaluations N).

5 Conclusion

A mathematical programming model was built for the stochastic resource allocation problem with time windows. A general encoding/decoding scheme was proposed for representing SRAPTW solutions. Constructive heuristics and random search algorithms were proposed to solve SRAPTW. Comparative experiments show that MMR-MP and RP-MP have obvious advantages over MMR-RS and RP-RS in most cases. MMR-MP and RP-MP share the similarity in the use of preset time under maximal probability strategy, which implies the superiority and necessity of utilizing the problem-specific knowledge embodied in Properties 3 and 4. In some instances, MMR-RS is the single winner, which reflects that MMR is an effective strategy for determining the order of resource-task pairs in allocation process. In contrast, RP-RS did not take the first place in any instances, and also performs the worst in most cases. The failure of RP-RS as compared with its competitors confirms that pure random search without utilization of problem knowledge is not a satisfactory choice for algorithm design. Generally, misuse of inaccurate knowledge in problems may cause the loss of optimality or lead to local optima. On the other hand, no use of problem-specific knowledge usually results in slow convergence of iterative search process. A delicate balance between the two aspects may bring better strategies for solving complex SRAPTW. On the basis of the proposed general solution representation scheme, how to design advanced neighborhood search operator or meta-heuristics [20–22] also deserves further studies in the future.