Discussion on “Dynamics model to simulate water and salt balance of Bosten Lake in Xinjiang, China” (Environmental Earth Sciences 2015; 74(3): 2499–2510)

Abstract

The paper titled “Dynamics model to simulate water and salt balance of Bosten Lake in Xinjiang, China” by Rusuli et al. is discussed. We identified a number of errors in the historical test of the system dynamics (SD) model proposed in the discussed paper and listed the completed principle regarding the historical test. The incomplete nature of the test may cause a deviation in the simulated result, and the conclusions based on the SD model may be incorrect. Thus, the historical test of the SD model must be improved. This discussion will have some reference value and ensure that similar errors are avoided when we analyse problems with system dynamics.

Introduction and description of the issues

Recently, the paper titled “Dynamics model to simulate water and salt balance of Bosten Lake in Xinjiang, China” (Rusuli et al. 2015) was published in Environmental Earth Sciences. The discussed paper established a system dynamics (SD) model to simulate the development of the water and salt balance system of Bosten Lake in Xinjiang, China, and the author offered several pieces of advice based on the simulated result. There are 23 variables in the SD model, two of which are level variables. The author established the SD model using the software “Stella”, and the feedback structure of the system can be described by the casual loop diagram.

To calibrate the SD model and reduce the uncertainty of the simulation progress, the historical test (Forrester and Senge 1980) of the SD model was conducted in the discussed paper. The variables “the lake level” and “the lake salinity” are selected as the examined variables, and a comparison between the simulated results and historical data from 1983 to 2013 was derived. The author used both graphical techniques and the ratio of the root mean square error (RMSE) to the standard division of measured data (RSR) to evaluate the performance of the SD model of water and salt balance during calibration.

In the discussed paper, the calculated RMSE error and RSR are low and acceptable in the simulated result, and the author concluded that the developed SD model can accurately simulate the dynamics of water and salt balance. However, we determined that there are missing variables in the casual loop diagram. It means that a deviation may present in the simulated result and not to be noticed. In particular, several missing variables are introduced as an important index to reflect the system’s behaviour. Thus, the validity of the SD model lacks credibility, and the conclusions based on the SD model may be incorrect. In summary, the historical test of the SD model proposed in the discussed paper must be further enhanced and improved.

Integrity of the historical test and the judgment method

Integrity of the historical test

To illustrate why there are errors in the historical test, the structure of the SD model is introduced as follows. According to the information feedback of the SD model, the basic structure of the system unit was divided into two categories: closed-loop structure and open-loop structure. The causal relationship of the feedback loop is formed in the closed-loop structure, and the causal relationship of directed style is formed in the open-loop structure. In addition, the former has the self-adjustment capacity in accordance with the information feedback (Fig. 1), while the latter does not have the capacity (Fig. 2). From Fig. 2, the connecting line of the open-loop structure is interrupted in the SD casual loop diagram, and missing variables are typically present in this structure.

Fig. 1

Variables’ causal relationship of the feedback loop and the related closed-loop structure. a Causal relationship of feedback loop, b closed-loop structure

Fig. 2

Variables’ passing causal relationship and the related open-loop structure. a Causal relationship of directed style, b open-loop structure

Determining the integrity of the historical test

In the casual loop diagram of the SD model, a directed line segment is a functional relationship between two interconnected variables. If the variable “V _i ” points to the variable “V _j ”, then V _j = f (V _i ). That is, “V _i ” can be used as an argument in the computation of “V _j ”, and the simulated result of the “V _i ” can pass to “V _j ” by the functional relationship (Oliva 2004). When the variable “V _j ” is selected as the examined variable, the interconnected variable “V _i ” will be checked and the error between simulated result and historical data will be derived. Moreover, the functional relationship between “V _i ” and “V _j ” will be tested. Evidenced by the same token, when the variables can reach the variable “V _j ” through certain directed path in the casual loop diagram, the simulated result of these variables will be delivered to “V _j ”. Thus, these variables and the functional relationships among them can be checked.

An easy method is described below to identify the integrity of the historical test. According to the causal loop diagram and the equations in the system, and starting from the examined variable, the tree graphs of the functional relationships among the variables of the SD model can be derived to list the variables that feed back the information to examined variables. The variable at the left of line segment is the variable “V _j ”, and the one at the right is the variable “V _i ” in the tree graph. In fact, the variable is included in the tree graphs when it can reach the examined variable through the directed path in the causal loop diagram. That is, the variable is listed in the tree graphs when it has been checked by the examined variable.

In the closed-loop structure, complete information feedback is included in the link of causation, and the system unit formed as a closed loop, except for the constant variables and exogenous variables. Based on the examined variable selected separately from the level variable, the rate variable, and the auxiliary variable, tree graphs of the functional relationships are derived, as shown in Fig. 3. Because all the variables in the closed-loop structure are correlative and restrictive to each other, they are included in the tree graph. In performing the historical test, checking any of the three variables could guarantee that all variables of the SD model will be checked.

Fig. 3

Tree graphs of functional relationships among examined variables to the other variables in the closed-loop structure. The examined variables are level variable (a), rate variable (b), and auxiliary variable (c)

However, the flow of the information feedback is interrupted in the open-loop structure, and not all variables are listed in the tree graph. From Fig. 4, the examined variable, which is selected from level variable and rate variable, cannot guarantee that all variables will be checked, except when the examined variable is the auxiliary variable. Thus, the variables that do not provide any information feedback to the other variables will be missing, and one cannot ensure the validity and reliability of the model. We usually do not find that the error appears in the simulated result of the system unit.

Fig. 4

Tree graphs of the functional relationships among the examined variables and the other variables in the open-loop structure. The examined variables are auxiliary variable (a), level variable (b), and rate variable (c)

The above analysis applies to the system’s structure in Fig. 5. According to certain decomposition rules, a large system can be divided into several subsystems. When there is information feedback among the subsystems, the whole system, except for the constant variables and exogenous variables, can form a closed feedback loop. Any variable in the feedback loop as the examined variable can guarantee the integrity of the historical test, except for the constant variables and exogenous variables. However, the subsystem, for which the information feedback loop is interrupted, could not be checked by the other subsystems. In this case, only when at least one appropriate variable is selected to check the other variables in the subsystem will the variables not be missed.

Fig. 5

Diagram of the system structure

Therefore, the judgement process of the integrity of the historical test can be divided into two steps. First, starting from the examined variables and along the direction of the information feedback, the tree graphs of the functional relationships among the examined variables and the other variables were derived. Thus, all variables in the tree graph are checked. Next, further judgement is required to determine whether the missing variables must be checked. If there are missing variables that must be checked, then the historical test is not complete and credible. In this case, at least one appropriate variable must be added to check the missing variables in the system. Only when all variables have been checked will the historical test be complete and credible.

Discussion

In the casual loop diagram of the SD model proposed in the discussed paper, there is an open-loop structure (Fig. 6). The variable “surface water salinity at Tashdian station” does not provide any information feedback to other variables in the system. In addition, the examined variable is not appropriate in this system unit. Therefore, it can be ascertained that there are missing variables in the system.

Fig. 6

Part of the open-loop structure in the casual loop diagram from Fig. 2 in the discussed paper

To verify the integrity of the historical test used in the paper, tree graphs of the functional relationships among the two examined variables and the other variables are derived (Fig. 7).

Fig. 7

Tree graphs of functional relationships among the examined variables “water volume of the lake” and “salt amount of the lake” and the other variables

From Fig. 7, not all the variables of the SD model in the discussed paper are included in the tree graphs. There are three missing variables: “water at Tashdian station”, “salt amount at Tashdian station”, and “surface water salinity at Tashdian station”. Through the analysis, these three missing variables are auxiliary variables and must be checked, or else the simulated result will produce a deviation. Unfortunately, the missing variables of “water at Tashdian station” and “surface water salinity at Tashdian station” are introduced as an important index to reflect the SD system’s behaviour. Thus, the historical test used in the paper lacks credibility, and the conclusions based on the SD model may be incorrect.

Note that, based on the method we proposed above, the variable “surface water salinity at Tashdian station” must be added as an examined variable at the very least. Only in this manner will all the variables be checked, thereby enabling the historical test to be considered as credible.

Conclusions

Based on the above discussion, we developed the following conclusions. In performing the historical test of their model in the discussed paper, there are three missing variables that must be checked: “water at Tashdian station”, “salt amount at Tashdian station”, and “surface water salinity at Tashdian station”. In addition, the missing variables of “water at Tashdian station” and “surface water salinity at Tashdian station” are introduced as an important index to reflect the SD system’s behaviour. Thus, the historical test used in the paper lacks credibility. The variable “surface water salinity at Tashdian station” must be added as an examined variable at the very least. Only in this manner will all of the variables be checked, thereby enabling the historical test to be deemed credible. In general, the historical test of the SD model proposed in the discussed paper must be further enhanced and improved.