Autonomous Decision Policies for Networks of Production Systems

Abstract

Modern production and logistic systems are facing increasing market dynamics: customers demand highly individualized goods, the adherence to due dates becomes critical and stipulated delivery times are decreasing. Particularly logistic networks, e.g. production networks or supply chains, are strongly affected by this trend. On the other hand, production networks have to deal with inherent internal dynamics, which are caused by e.g. machine breakdowns or rush orders. The concept of autonomous control, coming from the theory of self-organization, offers decentralized autonomous decision policies (ADPs), which enable logistic objects to make and execute decision on their own. Due to this kind of decision making, autonomous control aims at a distributed coping with dynamic complexity and, at the same time, at an improvement of the logistic performance. This contribution addresses the concept of autonomous control and the underlying autonomous decision policies as a novel concept for the control of the material flows in networks of coupled production facilities. Moreover, it shows different concepts of modeling and analysis of autonomously controlled networks. To achieve this goal, a dual approach including both, mathematical methods as well as simulation models, is presented. Subsequently, the possibilities to analyze the dynamic behavior of the autonomous logistic system are discussed, i.e., the system’s stability and its logistic performance. Finally, this contribution presents an exemplary case of a production network to demonstrate the practicability of the approach of modeling and analysis of autonomous control for production networks.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Modern logistic systems are exposed to various dynamically changing parameters in its internal and external environment. Especially logistic networks, e.g., production networks or whole supply chains, are affected by dynamical changes [53, 56]. For example, these dynamics are caused by customers’ increasing desires for individualized goods or the demand of decreasing delivery times and a strict adherence to due dates. Moreover, internal factors can cause unfavorable dynamic behavior of logistic networks, e.g., interdependencies between transportation and production processes or machine breakdowns. Manufacturing enterprises have to adapt to these changes rapidly. On the one hand, companies concentrate on their core competencies to sustain competitiveness. On the other hand they establish close cooperations with each other in order to satisfy the demand of their customers. In this context, several cooperation concepts for interconnected logistic networks were developed in the past. These concepts, for example virtual enterprises [7, 26] or production networks [57], aim at enabling companies to react promptly to dynamics. Related to this, several planning tasks for operating such networks occur in addition to classical production planning and control (PPC) functions. Comprehensible examples of these new tasks are the assignment of orders to production plants or the temporal coordination between transport and production processes. Especially the temporal coordination in geographically dispersed production networks gains importance [15, 40]. A lack of reconciliation between production and transport processes can lead to increasing throughput times, increasing tardiness of orders or underutilization of resources [28, 37]. Thus the integrated planning of transport and production processes has to ensure that an adequate quantity of raw material is supplied to the particular production plant at the right time. Furthermore, a high work-in-process (WIP) level should be avoided. A high level of WIP is unfavorable due to the resulting capital lockup. However, in highly dynamic and volatile situations centralized planning approaches, which solve the total planning problem incrementally, are not able to cope with occurring dynamics and unforeseen disturbances [21, 24]. Decentralized approaches, e.g., autonomous cooperating logistic processes, seem to be a suitable counterpart to classical centralized planning methods. This concept aims at enabling single logistic entities to make and execute operational decisions on their own. According to this idea, intelligent logistic objects (e.g., parts, machines or trucks) apply autonomous decision policies, in order to pursue their own logistic targets [59]. Due to the use of modern information and communication technologies (e.g., RFID, GSM, GPS, etc.) these objects are able to interact with others. Based on these interactions, logistic objects collect information about current local system states and use this information for decentralized decision making. Autonomous cooperating logistic processes aim at increasing the system’s robustness and its performance, due to autonomous and distributed decision making of intelligent logistic objects. The implementation of autonomous decision policies (ADPs) in production systems and entire production networks already showed promising results, in terms of an increased logistic target achievement and the robustness against disturbances [45, 46].

However, this kind of autonomous decision making causes a decentralized system behavior, which may affect the total logistic performance negatively or even leads to instability of the system [34, 60]. Roughly speaking, stability means that the state of a plant remains bounded over time, whereas instability of a network leads to infinite states. A network with increasingly growing WIP can be called unstable.

This contribution aims at explaining the idea of autonomous cooperating logistic processes and the fundamental concepts of ADPs in large-scale logistic networks to practitioners. In the beginning the theoretical background will be outlined briefly. The general focus of this contribution is set on describing how to implement ADPs in production networks and furthermore how to determine key-indicators of the systems using ADPs, such as logistic performance and stability. Therefore, this contribution is structured as follows (Fig. 1): In Sect. 2 a general definition of production networks is given, while Sect. 3 addresses operative planning problems of production networks. It discusses classical central approaches in this context. The concept of autonomously cooperating logistic processes and the underlying ADPs are presented in Sect. 4.

Fig. 1

Structure: ADPs for networks of production systems

Concrete approaches for modeling, simulating and analyzing the performance and the stability of ADPs in production networks are presented in Sect. 5. Subsequently, the application of these modeling and analysis approaches is presented in Sect. 6 in two examples of production network scenarios. Finally, Sect. 7 includes a summary and an outlook.

2 Production Networks

Relevant literature provides several definitions concerning networks of coupled interconnected production systems. In the context of this contribution the term production network is defined according to Wiendahl and Lutz [57], based on the orientation at the integrated planning of logistic processes: Production networks are company or cross-company owned networks of geographically dispersed production facilities. They focus on the mutual use of common resources and integrated planning of value adding processes in the network [57]. This allows achieving economies of scale through the joint planning and the common use of production resources. These types of networks may react promptly to internal or external disturbances due to redundancies of resources. An integrated view on production planning and transport planning requires additional tasks: Companies have to generate concepts for identifying new network partners, the network design and adjusting the PPC according to the network’s purpose [53]. However, this creates complex interdependencies between PPC of plants and coordination of transports, e.g., allocation problems between plants or planning of transport schedules and transport capacity [2, 40, 52]. Besides these operational planning problems, which concern a short-term time horizon, there are also several planning problems on the tactical and strategic level. The supply chain planning matrix, introduced by Meyr et al. [27], comprises all relevant planning problems for short-, mid- and long-term time horizons. It covers all dimensions of corporate logistics: procurement, production, distribution and sales. A classical problem of the long-term time horizon is the strategic network planning, with tasks like selection of strategic partnerships or localization of production plants [39]. In the mid-term time horizon, the so called master planning describes tasks of coordinating all procurement production, distribution and sales activities, which are necessary to fulfill the customers’ demands. The short-term time horizon concerns the operational level and contains classical tasks such as production planning and control. Furthermore, the operational level addresses the distribution and transport planning, as well as purchasing activities and material requirements planning. It is assumed that these tasks are implemented in software modules, which cover all of these planning problems [30]. Some authors argue that an integrated planning method, which solves all problems in an incremental way, may be challenged toughly by the occurrence of dynamics and unforeseen disturbances [21, 24]. Moreover, the structural complexity of large logistic networks is another limiting factor for the application of centralized optimization methods in the context of planning and operating such networks. At least single problems of production and transport logistics are NP hard [17]. Accordingly, optimal solutions can only be found for very small and simple instances in appropriate computational time. Thus, heuristics are commonly used for these kinds of scheduling problems. The following sections give a brief overview about different planning problems in large-scale logistic networks with a special focus on operative planning problems. These problem classes cover production logistics aspects, transport-related problems as well as integrated problem formulations.

3 Planning Problems in Production Networks

Sauer [40] describe planning tasks in supply or production networks as multi-site scheduling problems. Multi-site scheduling problems are an integrated formulation of production and transport problems, in terms of determining quantities and schedules for particular production facilities as well as determining transport schedules [9]. These approaches address three planning problems: the scheduling of the shop-floor, planning of transport operations and their coordination on the network level. The coordination in production networks comprises tasks of information updating of successors and predecessors. Dunbar and Desa [14] investigate in this context a distributed model adaptive control approach and compare it to a nominal feedback policy. They point out that this approach outperforms the nominal policy in situations with reliable demand forecasts. The MUST-architecture introduced by Sauer [40] describes an approach based on a central coordination instance, which creates a global schedule on the basis of locally generated schedules for all plants and the corresponding transportation activities. This global schedule takes the solutions of the sub-scheduling problems into account. Guinet [19] presents another centralized approach, which divides the total planning problem into sub-problems on the network level and on the shop-floor level. Shop-floor and transport problems will be characterized and described in the following.

3.1 Shop-Floor Problems

Shop-floor scheduling problems are a well-known problem class in operations research. The corresponding literature provides several comprehensive textbooks (e.g., [31] or [13, 35]. Hence, this section aims at giving a brief overview about different problem classes. Especially, the flexible flow shop problem will be discussed in detail, due to its realistic assumptions and its widespread application in analysis of autonomous controlled production systems.

General classification characteristics concerning shop-floor problems are: the machine types/the arrangement of machines, characteristic of jobs and objective functions. The machine types and the arrangement can be differentiated according to three main classes: single machine problems, multiple identical parallel machine problems and unrelated parallel machine problems [1, 36]. In contrast to single machine problems the class of multiple machine problems addresses the assignment of a job set to a set of multiple machines on one or more production stages. As a specification of parallel machine problems, unrelated parallel machines offer different processing times and setup times for different job types. As mentioned above, the flexible flow shop (FFS) problem is a special problem formulation of a shop-floor scheduling problem [22]. The FFS comprises a variable number of production stages, which contain a variable number of unrelated parallel machines per stage. Jobs running through the system have to pass each stage once. Due to the unrelatedness, the machines offer different process and setup times to the jobs. Algorithms for solving this problem type depend on the chosen logistic target system. Often the makespan is chosen as objective function. This means the timespan between the first order release time of the first job and the completion of the last job. Jungwattanakit et al. [22] propose multiple coupled algorithms which construct primarily a sequence for jobs on the first stage. Afterwards greedy algorithms assign the jobs to the machines on a stage. The greedy algorithm is repeated, until all jobs are assigned to stages. The contribution of Jungwattanakit et al. [22] shows that a combination of these algorithms with a genetic algorithm improves the optimization result.

The assumptions (different job types, unrelated parallel machines, variable number of resources, etc.) in the FFS problem formulation can be considered to be realistic and near to practice [1]. Thus, the FFS is often used for analyzing different autonomous decision policies in the production logistic context.

3.2 Transport Problems

The planning of transports in geographically dispersed networks is a complex task. Transport operation can be generally classified in short haul and long haul operations. Short haul operations describe the aggregation of different transport orders, which do not fully utilize the capacity of a transport carrier, to tours or round trips. Popular planning problems related to this area are the traveling sales man problem (TSP), the vehicle routing problem (VRP) or the pick-up and delivery problem (PDP) [54]. These problems and their derivatives focus on determining round tours starting and ending in one point (depot) for one or more transport carriers (trucks) to deliver a certain amount of goods to costumers.

Long haul planning addresses the delivery of goods over long distances with less nodes. Usually, in long haul transports line operations are implemented [16]. Thus, the particular transport route gets already fixed in advance and the transport operation takes place according to predefined policies. This type of transport initiation is commonly used in production networks. This planning problem can be divided in two sub-problems. The mid-term task of service network design includes the choice of a transport carrier (road, rail, sea, etc) and the circulation of the transport carriers [10]. The short-term planning aims at aggregating and assigning orders to loads. The triggering of transports in a long haul operation can be done by several policies. Usually, these transports are initiated in fixed frequencies according to a predefined schedule [18]. Another type is the so-called “go-when-full” policy. This policy implies that a truck starts a transport process, when a predefined loading quantity is reached [5]. In real word practice a mix-form of both can be found. This means, transports are initiated with predefined time windows, but within these time windows there is the possibility to operate with a go-when-full policy. The advantage of a go-when-full policy is an efficient utilization of the load carriers [10]. Their capacity is fully used in this case. A drawback in this kind of policy is the construction of loose schedules.

4 Autonomous Cooperating Logistic Processes

The idea of autonomous cooperating logistic processes is inspired by the theory of self organization. This section presents the definition of autonomous control and elaborates on autonomous decision policies.

4.1 Definition

According to the collaborative research center 637 “Autonomous cooperating Logistic Processes: A Paradigm Shift and its Limitations”, the following definition of autonomous cooperating logistic systems is given: “Autonomous control describes processes of decentralized decision-making in heterarchical structures. It presumes interacting elements in non-deterministic systems, which possess the capability and possibility to render decisions independently. The objective of autonomous control is the achievement of increased robustness and positive emergence of the total system due to distributed and flexible coping with dynamics and complexity” [59]. According to this definition autonomous control is characterized by a shift of decision-making capabilities from the total system to its elements, which allows intelligent logistic objects to route themselves through a logistic network according to their own objectives [60]. In the context of this definition intelligent logistic object may be either physical objects (e.g., trucks, machines, etc.) or immaterial objects (e.g., production orders or transport orders). Modern information and communication technologies can provide an infrastructure, which enables an exchange of information about current local system states between these objects. On this basis the objects are able to generate decisions according to different autonomous decision policies. Due to these multiple decentralized decisions the local and the global behavior should be influenced in a positive manner, for example, in terms of improving the handling of dynamics caused by unforeseen events (e.g., machine breakdowns) [60].

In the past, ADPs have been developed for all areas of the logistic chain: There exist ADPs for transportation and route planning (e.g., [38]), production logistics (e.g., [43]), transport collaborations [6], or production networks (e.g., [45]). In the following, different ADPs for production systems and production networks are presented.

4.2 Autonomous Decision Policies

Generally, ADPs enable decision making of intelligent logistic objects. In the context of production systems and networks all existing ADPs facilitate decision making of parts or jobs (semi-finished products), to decide about possible routes through the system. Scholz-Reiter et al. [48] propose a classification of ADPs according to local information methods and information discovery methods. Information discovery methods, i.e., the distributed logistics routing protocol (DLRP), collect information from other objects. The DLRP is inspired by communication protocols of wireless ad hoc networks. Intelligent logistic objects using the DLRP send requests into the logistic network. By receiving replies, the object collects information about the system, which is used for local decision making. This discovery does not cover the whole system, but it is directed to information that is relevant for the actual decision. The DLRP is designed for production environments [50] as well as for transport logistic routing problems [38]. However, this contribution focuses on local information methods. Local information methods enable jobs to decide about further processing steps. Jobs using one of these methods only gather local information about states of direct succeeding buffers and machines.

According to a classification introduced by Windt and Becker [58] local information methods can be further divided into rational policies, bounded rational strategies and mixed forms. Rational strategies use solely rational measures (e.g., throughput times or due dates) for the decision-making process. In contrast, biologically inspired strategies which belong to the class of bounded rational strategies, try to transfer mechanisms from biological self-organizing systems to the decision-making in production networks. Table 1 presents different ADPs, which can be applied to production networks. It differentiates between shop-floor related and network-related strategies and presents their main characteristics as well as a short overview about the algorithmic scheme.

Table 1 Autonomous decision policies for production networks

Table 2 Weighted adjacency matrix

The QLE policy enables parts in a production system to estimate the waiting and processing times of different alternative processing resources. It uses exclusively local information to evaluate the states of the alternatives. The application of this policy leads to a better system performance regarding throughput times compared to classical scheduling algorithms in highly dynamic situations [50].

Similar to the QLE, the DUE policy estimates waiting and processing times. While the QLE uses this information for minimizing part-related throughput times, the DUE policy orientates at the tardiness of parts. A part using this policy decides for an alternative resource which offers the lowest difference between estimated due date and pre-planned due date [47].

In contrast, the PHE policy is a bio-inspired strategy. The approach is based on the idea to imitate the process of ants marking possible routes to food sources. Ants leave pheromone marks between the nest and food sources. Other ants can detect those pheromones and will follow the trail with the highest concentration of pheromones [32, 33]. This is transferred to logistic systems: During the production process, the parts leave information about their processing and waiting times at a corresponding machine. Following parts entering a stage of the shop-floor compare this artificial pheromone concentration by computing average value of the waiting time data of the last five parts and choose a production line. Thus, the pheromone concentration depends on waiting and processing times of previous parts. To model the evaporation process of natural pheromones a moving average of waiting time data is used [3].

The honey bee algorithm (HBA) is another bio-inspired strategy. It uses the foraging mechanisms of honey bees’ colonies. In nature bees advertise possible food sources with a so-called ‘waggle dance’. The duration of this dance depends on the ratio between energy consumption of the flight (between hive and food source) and available energy of the source. The probability of bees recognizing the dance of a dancing bee is proportional to the dancing duration. According to this principle parts are able to advertise different alternative production resources by means of the machining quality, which is determined by calculation of the benefit provided by a particular machine and the throughput time needed for this step [44].

The natural process, which inspires the CHE policy, differs from the PHE and the HBA policy. It is not inspired by coordination principals of social insects, but on movement processes coming from micro-biology. Natural bacteria are able to direct their movement according to the concentration of attractants (e.g., food substances) or repellants (e.g., toxic substances). Therefore, bacteria perform a random biased walk to find appropriate food sources. This basic movement principle is transferred to autonomous decision making by the CHE policy. Parts using this policy decide according to the gradient of logistic target values of different decision alternatives [49].

All ADPs described above were implemented in the past for several production logistic scenarios. In general, these policies can also be used for the decision making on the network level. Currently, the QLE and the PHE policy have already been transferred to decision making on the network level. The nQLE enables the decision making on the network level similar to the QLE policy. The network-related version enables an allocation decision of parts to plants. Therefore, the nQLE estimates, similar to the QLE policy on the shop-floor level, the transport duration from one plant to the next and estimates the processing times in the respective plant. The part chooses the plant with the lowest estimated transport and processing times. The nPHE policy is based on the same principles as the PHE policy. Intelligent parts choose one of alternative succeeding plants according to information about the processing and waiting times of previous parts. In contrast to the PHE policy this information is not limited to the waiting times at the next machine, but is based on the time spent to pass the transport system and the corresponding plant [50]. After processing in one plant the part leaves this information as an artificial pheromone at the plant, which can be detected by the following parts.

Concerning ADPs, Scholz-Reiter et al. [42] present a framework for choosing the right policy for a particular production scenario. The underlying evaluation of this framework applies evaluation methods, which will be presented in Sect. 5. This contribution presents tools for evaluation of ADPs in production networks.

5 Modeling and Analysis of ADPs in Production Networks

When dealing with production networks, it is self-evident to consider an integrated modeling of production networks that covers both job-shop scheduling and transport logistic problems from an integrated point of view. Such an integrated modeling approach for production networks is presented in Sect. 5.1. The representation of time in models of production networks varies throughout the literature: it can be distinguished between discrete event and continuous simulation models [29]. Sections 5.2 and 5.3 present different modeling and simulation approaches, i.e., a mathematical approach and a discrete event simulation (DES) in order to validate the obtained simulation results concerning ADPs in production networks against each other. Production networks usually have to deal with dynamic variations, which can be caused by internal factors or by the (external) environment. Hence, a pure static analysis of logistic performance indicators seems to be not sufficient to cover the effects and the interdependencies of these dynamics. Thus, Sect. 5.4 presents appropriate measures for analyzing production networks.

5.1 Integrated Modeling of Production Networks

For the purpose of analyzing autonomously controlled production networks a matrix-like production network scenario was introduced by Scholz-Reiter et al. [45]. This matrix-like model allows the analysis of the dynamical behavior, the stability and the logistic performance of a multi echelon production network with detailed shop-floor and transport models.

Figure 2 shows the generic structure of this model. It consists of a variable number of network stages, which comprise a variable number of production plants per stage. Furthermore, these production plants are represented as a shop-floor scenario. Each of these shop-floors is a matrix-like model (similar to Scholz-Reiter et al. 2005).

Accordingly, different production resources (buffers and machines) are located on a variable number of production stages. Figure 2 depicts this relation. Transport systems connect the production plants on the network level with each other. The network is able to process different job types. The arrival rate u(t) describes the input of jobs to the network as a function of time. In order to model different demand situations this function can be modeled as an arbitrary mathematical function. For example, a sinusoidal function can be chosen for modeling seasonal demand fluctuations (similar to [43, 45]). However, stochastic inputs can be chosen as well. All transports in this model are direct deliveries, in terms of a door-to-door delivery. This means that each transport between two plants is initiated and operated separately. The model allows integrating different direct transport strategies, like a “go-when-full-policy” or a “frequency-based-policy” with pre-defined departure times as described by Crainic [10]. Trucks using the “go-when-full-policy” will depart at a particular plant, whenever their total load capacity qis reached. In a “frequency-based-policy” all departure times are predefined and scheduled. This means that the transportation starts at a certain time, which consequently may lead to variations in the trucks load quantity q. Both transportation policies are commonly used in door-to-door transports in long haul operations.

Fig. 2

Generic production network model with kxj plants and mxn machines per plant (see [45])

The modeling of particular scenarios can be done by different modeling tools, depending on the purpose of the analysis. This contribution introduces a mathematical approach and an engineering-orientated simulation approach. The formulation of a mathematical model, which is based on differential equations, is a suitable approach to analyze the stability of a production network [12]. Moreover, the engineering-orientated simulation approach can be used to refine and validate the results of a mathematical stability analysis. It can be used to analyze the logistic performance of ADPs in production network.

5.2 Mathematical Modeling and Stability Analysis

In order to analyze and to make statements about the dynamics of production networks, the mathematical modeling by differential equations [20] can be used, which can be called the macroscopic view. Each plant of a production network is called subsystem. General production networks consist of n subsystems and each subsystem is modeled by a differential equation that describes the incoming and outgoing material or information flows as follows:

$$ \dot {x}_i (t)=\sum_{j=1,j\neq i}^n {c_{ij} (x(t))\tilde {f}_j (} x_j (t))+u_i (t)-\tilde {c}_{ii} (x(t))\tilde {f}_i (x_i (t)),\quad i=1,...,n. $$

Here $x_{i}$ denotes the state of the ith subsystem and is a positive real value. The state can be interpreted as the number of unprocessed parts of a subsystem, but one can choose any other state variable such as the number of unsatisfied orders for example. The state of the whole network is denoted by $x=(x_{1} ,{\ldots},x_{n})^{T}\!.$ The positive real value t denotes the time of the system. The term $u_{i}$ represents an external input into the subsystem, e.g., supply of raw material.

Each plant processes the material with a production rate $\tilde {c}_{ii} (x(t))\tilde{f}_i (x_i (t))$, where $\tilde {c}_{ii} \in {\mathbb{R}}_+$ and $\tilde {f}_i (x_i (t))$ is a continuous, positive definite and monotone increasing function. The processed material is sent to the jth subsystem of the network with the rate $c_{ji} (x(t))\tilde {f}_i (x_i (t)),j\neq i$ where $c_{ji}$ is a positive real value and represents a distribution parameter for processed material from subsystem i to j. The term $\sum_{j=1,j\neq i}^n {c_{ij} (x(t))\tilde {f}_j (x_j (t))}$ is the internal input of material from other subsystems to the subsystem i.

Denoting $c_{ii} :=-\tilde {c}_{ii}$ the above equations can be rewritten as an interconnected system, which represents the whole network in a vector form

$$ \dot {x}(t)=C(x(t))\tilde {f}(x(t))+u(t), $$

(1)

where $\tilde {f}(x(t))\!=\!(\tilde {f}_1 (x_1 (t)),...,\tilde {f}_n (x_n (t)))^{T},u=(u_1 ,...,u_n )^{T},C(x(t))= (c_{ij} (x(t)))_{nxn}.$

ADPs are modeled by the production rate $\tilde {c}_{ii} (x(t))\tilde {f}_i (x_i (t))_{ }$ and the distribution coefficients $c_{ij}.$ The production rate depends on the state of a subsystem and the ability to adapt the production speed of a plant in a network can be modeled by an appropriate choice of this rate. Namely, if there is a lot of unprocessed material, the plant increases the production and, conversely, if there is less unprocessed material the production speed goes down. For example, one can choose $\tilde {f}_i (x_i (t))=x_i^2$ or $\tilde {f}_i (x_i (t))=(1-\exp (-x_i )).$

By the distribution coefficients $c_{ij}$ a centralized or decentralized planning scenario can be modeled, where constant coefficients are identified as central planning. For example, the nQLE, nPHE or other ADPs can be implemented by the following choices of the coefficients $c_{ij}\!:$

• for the nQLE

  $$ c_{ij} :=\frac{\frac{1}{x_i +\varepsilon }}{\sum_k {\frac{1}{x_k +\varepsilon }} }, $$

• for the nPHE

  $$ c_{ij} :=(1-{\it v}_i )\frac{\tilde {f}_i (x_i )}{\sum_k {\tilde {f}_k (x_k )+\varepsilon } }+\sum_{k\neq i} {{\it v}_k \frac{\tilde {f}_k (x_k )}{\sum_q {\tilde {f}_q (x_q )+\varepsilon } }} , $$

• for an integrated ADP

  $$ c_{ij} :=\frac{\frac{\tilde {f}_i (x_i )}{x_i +\varepsilon }}{\sum_k {\frac{\tilde {f}_k (x_k )}{x_k +\varepsilon }} }, $$

where k and q are the indices of the subsystems, which get material from subsystem j, ${\it v}_{i}$ are evaporation constants, $i=1,{\ldots},n$ and $\varepsilon$ is a positive constant to assure that the term $c_{ij}$ is well-defined.

So far, one important circumstance that occurs in production networks has been left out: transportation times of material from one plant to another. These transportation times can be modeled using time-delay systems as follows:

$$ \dot {x}_i (t)=\sum_{j=1,j\neq k}^n {c_{ij} (t)\tilde {f}_i (x_j (t-\tau _{ij}))+u_i (t)-\tilde {c}_{ii} } (t)\tilde {f}_i (x_i (t)), \quad i=1,...,n. $$

(2)

Transportation times are represented as time-delays $\tau _{ij} \in {\mathbb{R}}_+$, which denote the time needed for transportation from subsystem j to i. In Eq. (2) the time-delays are included in the terms which represent the inflow of material from other subsystems, where $c_{ij}$ can also depend on a time-delay. In the terms which represent the external input and the internal production rate no insertion of time-delays is necessary.

The consideration of transportation times makes the analysis of production networks more complex, but more realistic too. Due to the abstract level of this view, the model (1) or (2) is used to analyze the dynamics and make general statements about the dynamics of production networks from a macroscopic view. The results can be used to adapt the simulation model and, conversely, the results can be refined by results of the simulation view. This will be explained with examples in Sect. 5.4.

5.3 Simulation Models

Simulation approaches are often used for the analysis of stability of production networks in order to refine the mathematically found stability regions. Furthermore, they can be used to investigate different system aspects like logistic target achievement for time-varying systems parameters. For the analysis of ADPs in production networks, several simulation approaches were used in the past (e.g., [43]). These simulation models can be classified according to their general function principle concerning the representation of time in the simulation model. Continuous time simulations represent the time of the simulation model as a real variable. All system states in the simulation model change with dependence on the simulation time variable. In contrast to this, in a discrete time simulation model the time elapses in predefined equidistant time steps. A particular variant of discrete time models is discrete event simulation models [4, 62]. Here, the time elapses in non-equidistant time steps. The states of the simulation model change according to events. In the production logistic context these events describe the arrival of raw material in a source or the end of a particular production step [25], for example. Besides the representation of time, some authors discuss the purpose of analysis as a possible classification characteristic. Morecroft and Robinson [29] describe differences in the modeling representation and the interpretation between discrete event and continuous simulation models. Accordingly, discrete event simulations are applied for the representation of very detailed scenarios, which should be investigated with regard to the interaction of single system elements, while continuous simulations are used for investigations of general dynamic aspects of the system [8, 29]. The usage of different modeling and simulation approaches helps to validate the obtained simulation results concerning ADPs in production networks against each other. Possible differences or mistakes can be detected easily. A comprehensible example for this approach is the determination of stability regions of autonomously controlled production networks. In Scholz-Reiter et al. [51] a discrete event simulation approach is used for the refinement procedure.

5.4 Measures for Analysis

Production networks are steadily exposed to dynamic variations, caused by internal reasons and by the external environment. Hence, a pure static analysis of logistic performance indicators seems to be not sufficient to cover the effects and the interdependencies of these dynamics. Nevertheless, classical logistic performance indicators should not be neglected. According to Wiendahl [56] the logistic key performance measures are throughput time (TPT), delivery liability, work-in-process (WIP) and utilization. The throughput time is the time-span spent by a particular product in a production system. From a customer’s point of view short TPTs are desirable, due to the shorter possible delivery times. Another aspect of this customer’s perspective is the delivery reliability, which means a delivery of goods to the customer at the right time in the right quantity. The performance indicators WIP and utilization belong to the logistic costs. A high level of WIP means that the buffers of the system are filled with numerous semi-finished goods and raw material. This leads consequently to high degree of capital lock-up. From an economical point of view the WIP should be at a low level, while the system is fully utilized.

Windt et al. (2008) developed a vector-based approach which allows one to weight these targets according to the subjective preferences and to aggregate these weighted targets in one performance indicator, called logistic target achievement. This value depends on pre-defined targets, the operative target achievement and weight factors. The total logistic target achievement gives information about the performance as a percentage value. By applying this approach, different configurations of production systems can be compared easily. For the objective of the analysis of autonomously controlled production systems, this vector-based approach can be used to compare different ADPs in a defined way (e.g., [46]). Furthermore, Windt et al. [58] introduce an autonomous control application matrix (ACAM), which proposes an evaluation of different scenarios with ADPs on the basis of this vector-based approach.

Additionally, the identification of stability regions is generally crucial for planning and operating logistic networks as an aspect of the dynamic systems behavior. In this context, mathematical models are often used to determine stability regions. Typical examples of unstable behavior are unbounded growth of unsatisfied orders or unbounded growth of the queue of the workload to be processed by a plant or a machine. This causes high inventory costs and loss of customers. To avoid instability of a network it is worth to investigate its behavior in advance.

Stability means, roughly speaking, that the number of unsatisfied orders or the number of unprocessed parts remains bounded over time. More precisely, the local input-to-state stability (LISS) property from control theory is used and by the means of LISS the state of a system can be estimated. More details about this property can be found in Dashkovskiy and Rüffer [11].

A useful tool to verify the LISS property of a system is a Lyapunov function, which is positive definite and radially unbounded and can be interpreted as the energy of the systems state. A LISS Lyapunov function ${\it V}_{i}$ of the ith subsystem has the property that if ${\it V}_i (x_i )\geq \max \left\{ {\max _{j\neq i} \gamma _{ij} (V_j (x_j )),\gamma _i (\left| {u_i } \right|)} \right\}$ holds, where the gains $\gamma_{ij}$ and $\gamma_{i}$ are positive definite, zero at zero and strictly increasing functions, then the energy decreases. If ${\it V}_i (x_i )<\max \left\{ {\max _{j\neq i} \gamma _{ij} (V_j (x_j )),\gamma _i (\left| {u_i } \right|)} \right\}$ then the energy of the system is bounded by the expression on the right side of the previous inequality. Overall, the trajectory of a system is bounded. More details can be found in [11].

By the gains, statements about the behavior of the system can be made. For example, they offer information about the upper bound of the trajectory of a system or in other words about the highest inventory level of a system. This information is helpful for plant owners, because they can plan the size of the inventory in advance and they can also design their plant in a way to assure stability.

The tool of a Lyapunov function can be used for the stability analysis in the following way:

Consider a network consisting of n subsystems and assume that each subsystem has a LISS Lyapunov function, i.e., each subsystem has the LISS property. Then, the overall network has the LISS property provided that the small-gain condition (SGC) is satisfied (see [11]).

Simply speaking, the SGC states that along every existing circle in the network the composition of the corresponding gains is less than the identity (see [11]).

Concluding this, to verify if a system is stable, one has to find LISS Lyapunov functions for the subsystems, the corresponding gains and to check the SGC, then stability is verified. Otherwise, one has to find other LISS Lyapunov functions candidates and gains. If all efforts are not successful, then no statement about stability is possible.

To assure the stability of a network by using the properties of the Lyapunov functions and the SGC one gets conditions on relevant system parameters, as the production or distribution rates and the external inputs. Using the model (1) in Sect. 5.2 describing general production networks and assuming that the distribution coefficients are bounded, the following condition for unbounded production rates can be derived:

If there exist $a\in {\mathbb{R}}^{n},a_i >0$and $\varepsilon \in {\mathbb{R}}^{n},\varepsilon _i >0,\quad i=1,...,n$, such that

$$ C(t)a<\varepsilon $$

holds, then the whole network (1) has the ISS property, which is the global variant of LISS.

For production rates, which are bounded up to a certain limit $\alpha _i :=\sup _{x_i } \left\{ {\tilde {f}_i (x_i )} \right\}$ the condition

$$ C(t)\alpha +\left\| u \right\|_\infty <\varepsilon , $$

can be derived to assure that a network has the LISS property, where $\alpha =(\alpha _1 ,...\alpha _n )^{T}$ and $\left\| u \right\|_\infty$ denotes the essential supremum norm of the external input. Taking transportation times into account one gets similar conditions to assure stability of a production network modeled by the equations as in (2).

These conditions form a stability region: for parameter constellations (i.e., set of parameters) within this region, stability is guaranteed. For parameter constellations outside this region the tool of a Lyapunov function does not offer a statement about stability. At this stage, simulations are performed to refine the stability region.

By the analysis using Lyapunov functions a large set of parameter constellations which assure stability are identified. Only few parameter constellations have to be simulated. To identify stable or unstable parameter constellations a truncation criterion needs to be defined. Then, the simulation results refine the stability region.

This dual approach using the analytical and the simulation model has the advantage of less time consumption to identify stability regions in contrast to a pure simulation approach. This is especially relevant since the time needed for a simulation run increases exponentially as the number of plants, links and parts in a network are increased. To identify parameter constellations which assure stability and to make statements about the inventory levels of the plants of a complex network with a large number of plants is a problem which cannot be solved in an acceptable time. The dual approach presented helps to derive and refine parameter constellations in reasonable time by assuring stability (see [51]) and is presented in the following Fig. 3, where a scheme of a stability analysis is displayed.

Fig. 3

Scheme of the stability analysis (similar to [51])

6 Examples of ADP Implementation

This section presents the approach of modeling and analyzing autonomous decision policies in two exemplary cases of production networks. The first focuses on stability analysis and a refinement of stability regions of a relatively simple network. The second is used to demonstrate the performance evaluation of different autonomous decision strategies (for the structure see Fig. 4). The first network consists of three plants, the second has six plants. In the first example a macroscopic view is considered; the second example is investigated in detail representing the shop-floor of the plants, consisting of $3\,\times \,3$ machines (see Fig. 4).

6.1 Stability Analysis

Fig. 4

a Production network scenario with three plants. b Production network scenario with six plants

According to Fig.  4a the material flow between the plants is defined as follows: The input of raw material arrives at plant 1 and plant 3. All material produced in plant 1 is delivered to plant 2. From here 50% of the goods are delivered to the customers and 50% for further processing to plant 3. Plant 3 sends 50% of its output to plant 1 and plant 2 each. In order to model seasonal demand fluctuations both inputs to plant 1 and plant 3 are modeled as a sinusoidal function $u_{i}(t){:}$

$$ u_i (t):=AV_i \cdot (\sin (t)+1)+5,\quad i=1,3. $$

The parameter $AV_{i}$ determines the intensity of the fluctuations in terms of the amplitude. In this example this parameter is used to generate different input situations. The plants are able to decide autonomously about their current production rate $\tilde {f}_i$ at the time point t. It is assumed that this decision depends on the actual workload in the following form:

$$ \tilde {f}_i (x_i (t)):=\alpha _i (1-\exp (-x_i (t))),\quad i=1,2,3.,\quad {\alpha _i \in {\mathbb{R}}_+} $$

This means that the production rate of plant i depends on the WIP of the plant. According to this equation, the production rate will be very low in cases of low WIP in the plant. Otherwise the production rate will be set to its maximum $\alpha_{i}$ for a high WIP level.

This example aims at determining the lowest possible values of $\alpha_{i}$ or a certain input situation, which depends on $AV_{i}.$ To do so, the internal structure of the plants is neglected in a first step. Stability conditions will be derived only macroscopically on the network level. These results will be subsequently refined by a simulation model.

In this first step the network is modeled by differential equations:

$$ \begin{array}{l} \dot {x}_1 (t)=u_1 (t)+0.5\cdot \tilde {f}_3 (x_3 (t))-\tilde {f}_1 (x_1 (t)), \\ \dot {x}_2 (t)=\tilde {f}_1 (x_1 (t))+0.5\cdot \tilde {f}_3 (x_3 (t))-\tilde {f}_2 (x_2 (t)), \\ \dot {x}_3 (t)=u_3 (t)+0.5\cdot \tilde {f}_2 (x_2 (t))-\tilde {f}_3 (x_3 (t)). \\ \end{array} $$

These equations describe the change of WIP in the three plants and consider the transport connections and quantities as well as the current production rate of a plant. According to the scheme in Fig. 3 the next step is to derive stability conditions using Lyapunov functions and gains. A very detailed technical description for this can be found in Scholz-Reiter et al. [51]. For this network the following stability conditions can be calculated:

$$ \begin{array}{lll} \alpha_1 >0.5\cdot \alpha_3 +\max_t \{u_1 (t)\},& \alpha_2 >0.5\cdot \alpha_3 +\alpha_1,& \alpha_3 >0.5\cdot \alpha_2 +\max_t \{u_3 (t)\}. \\ \end{array} $$

By solving this system of inequalities, the maximal production rate for that stability can be guaranteed and can be calculated for the corresponding value of $AV_i.$ For example, choosing $AV_i \equiv 5_{ }$ leads to:

$$ \begin{array}{lll} \alpha _1 >37.5,& \alpha _2 >60,& \alpha _3 >40. \\ \end{array} $$

From a mathematical point of view the stability of the production network can be guaranteed for the values indicated. This does not mean that the network is unstable for values that violate these inequalities. To illustrate this, a continuous simulation of the differential equation model is conducted.

For different values of $AV_{i}$ the production rate of all plants is reduced stepwise in several simulation runs. The simulation model is considered to be unstable, whenever the WIP of a plant starts to rise continuously about 10% in a time period of 30 days. Figure 5 depicts the results of the simulation model for plant 3 and compares it with the calculated results.

Fig. 5

Calculated and simulated stability regions for plant three

These results show that the simulation model is still stable in this case, even if the production rate is below the calculated stability boundary. Using simulations, the calculated stability region can thus be refined.

Figure 6 clarifies this. It presents simulation results for different values of $\alpha_{i}$ and $AV_i \equiv 5$ taken from the calculated stability region, the simulated stability region and the simulated instability region. The WIP remains bounded over time for the calculated stability region and the simulated stability region. By contrast the WIP grows continuously in plant P $_{1}$, P $_{2}$ and plant P $_{3}.$ The WIP in P1 increases continuously about 0.049, in P $_{2 }$ about 0.02 and in P $_{3 }$ about 0.029 units per time unit.

According to the results presented in Fig. 6, the mathematical determination of a stability region provides a good starting point for the refinement. Hence, this approach allows the identification of the border of the stability region with less time efforts than a pure trial and error simulation approach.

Fig. 6

WIP against time for examples of the calculated stability region, the simulated stability region and the simulated instability

In general, different simulation approaches can be applied for the refinement. Scholz-Reiter et al. [51] successfully applied a discrete event simulation and a continuous time simulation based on differential equations for the refinement of stability regions. It was shown that the refinement results of both simulation approaches provide similar stability results.

6.2 Implementation and Evaluation of Different Autonomous Decision Policies

The second example presents the modeling and implementation of different ADPs on the shop-floor and on the network level. Therefore, the network depicted in Fig. 4b is considered. In order to keep this example simple, only one logistic target measure is considered. This example focuses on the total throughput time, which denotes the time spent by the parts to pass through the entire network.

This scenario has six different plants on four network stages. On stage one and on stage four there is only one plant. On stage two and three there are two parallel plants each. Additionally, every plant consists of a shop-floor with $3 \times 3$ machines. The distances between the plants are summarized in Table 2.

The transports between plants are triggered by a “frequency-based” policy. This means that a transport starts in pre-defined time intervals. The interval in this example is set to 15 h.

There are three different job types in this scenario. These job types differ in their processing times on the shop-floor level in every plant. The processing times are summarized in Table 3.

As in the first example, the arrival rate of jobs in this scenario is set to a sine function in order to model demand fluctuations:

$$ u(t)=\lambda +AV\cdot \sin (t+\varphi ) $$

This function has a phase shift $\varphi=1/3$ of a period for each job type, so that the maximal arrival rates of all job types are not simultaneous. The variable $\lambda$ defines the mean arrival rate and is set to 0.4 1/h in all simulation runs. The second variable AV determines the intensity of the arrival rate fluctuation, as in example 1, AV is set to 0.125 1/h.

The purpose of this example is to describe how to choose an applicable combination of different ADPs for this particular network configuration. Therefore, the ADPs QLE, PHE, nQLE and nPHE are implemented on the shop-floor and the network level to a simulation model, exemplarily. Table 4 shows the different combinations of ADPs and summarizes the results of the simulation runs in a form which is comparable to the ACAM described above.

Table 3 Processing times (h:mm)

Table 4 Simulation results

This relatively simple example demonstrates that a detailed analysis of different combination of network-related and shop-floor-related ADPs is necessary. In this example the combination of QLE and nPHE performs best with respect to minimizing the total throughput time (TTPT), which is the time span needed by a part to pass through the entire network. On the other hand the combination of PHE and nPHE seems not to be suitable for this particular network. This combination leads to the highest mean TTPT.

Figure 7 depicts these results in more detail. It presents the TTPT against the simulation time of each possible combination. The nPHE leads to smoother patterns in the TTPT compared to the combination of the QLE/nQLE (Fig. 7a, 7b). A comparison of the corresponding standard deviations confirms this. In combination with the shop-floor-related QLE method the best result of mean TTPT (85.84 h) is realized.

Fig. 7

Total throughput time against simulation time: a combination QLE/nQLE, b combination QLE/nPHE, c combination PHE/nPHE, d combination PHE/nPHE

A similar curve shape is observed for the combination of the PHE/nQLE, but the absolute values differ: in this case the mean TTPT is 21.89% higher than in the first case. This can be explained by the time horizon used by the methods. The pheromone-based method uses data from past events, while the QLE is based on actual information. In the case at hand, the data on the shop-floor level used by the PHE method does not represent the current situation properly, which leads to unsuitable autonomous decisions on the shop-floor level and consequently to a longer TTPT.

The results of the network-related pheromone-based approach are different. In combination with the shop-floor-related QLE method the lowest mean TTPT (85.84 h) is realized.

Figure 7d presents the TTPT of the PHE/nPHE combination. Again, the shop-floor-related pheromone-based method leads to high throughput times in the plants, which corresponds to the effects discussed concerning the PHE/nQLE combination. Additionally to this, the nPHE method uses the information of the throughput times in the plants for the autonomous decision making on the network level. Due to the time-varying and imprecise information allocation decisions made by the nPHE method are not suitable in this situation. Consequently, this combination leads to the highest TTPT value.

This example illustrates the potentials of the application of autonomous decision policies in production networks. Combined autonomous decision policies on the network level and on the shop-floor level may lead to an acceptable logistic performance. However, the underlying dynamics and their consequences should not be neglected. In the case at hand the combination of the PHE/nPHE method leads to a dynamic interplay between network and shop-floor-related decisions which are undesirable and consequently decreases the logistic performance. Thus, the design and implementation of autonomous control strategies in production networks should be integrated in an intensive analysis of relevant system properties such as stability and systems performance.

7 Summary

This contribution described the integrated coordination between production and transport processes as an essential task of operating production networks. Different central planning and scheduling functions for shop-floor and transport operations were presented. In this context, different ADPs and the concept of autonomous cooperating processes were introduced as a novel approach to coordinated logistic processes in production networks. Several ADPs were introduced and described. Additionally, approaches for modeling and analyzing ADPs in production networks were presented and discussed in mathematical terms and via simulative approaches. Based on the mathematical modeling approach, criteria for the stability of production networks can be derived, which subsequently can be refined by simulations. Finally, two examples for analyzing the stability and the performance of autonomous decision policies in production networks were given.