Boon and Bane: On the Role of Adjustable Parameters in Simulation Models

Abstract

We claim that adjustable parameters play a crucial role in building and applying simulation models. We analyze that role and illustrate our findings using examples from equations of state in thermodynamics. In building simulation models, two types of experiments, namely, simulation and classical experiments, interact in a feedback loop, in which model parameters are adjusted. A critical discussion of how adjustable parameters function shows that they are boon and bane of simulation. They help to enlarge the scope of simulation far beyond what can be determined by theoretical knowledge, but at the same time undercut the epistemic value of simulation models.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Simulation brings together the important notions of model, theory, and experiment. Each of these notions has been discussed extensively in the philosophy of science. Consequently, the philosophy of simulation debates whether and how these established conceptions have changed with the rise of simulation technology.¹ We do not enter the discussion of what, in that context, an adequate conception of a simulation experiment is, nor what an appropriate notion of a theoretical model is. Instead, we focus on the interface of model and experiment. Here, adjustable parameters enter the picture. They might appear as a minor detail, a technical matter of smoothing out imperfections of a model. However, we argue that they are of central importance in simulation methodology, though they are a two-edged affair. They help to enlarge the scope of simulation far beyond what can be determined by theoretical knowledge, but at the same time undercut the epistemic value of simulation models. In short, adjustable parameters are boon and bane of simulation models.

Let us motivate this claim in more detail. Experimentation is a key element when characterizing simulation modeling,² exactly because it occurs in two varieties. The first variety has been called theoretical model, computer, or numerical experiments. We prefer to call them simulation experiments. They are used to investigate the behavior of models. Clearly simulation offers new possibilities for conducting experiments of this sort and hence investigating models beyond what is tractable by theoretical analysis. We are interested in how simulation experiments function in simulation modeling. Importantly, relevant properties of simulation models can be known only by simulation experiments.³ There are two immediate and important consequences. First, simulation experiments are unavoidable in simulation modeling. Second, when researchers construct a model and want to find out how possible elaborations of the current version perform, they will have to conduct repeated experiments.

The second variety is the experiment in the classical sense. When comparing simulations to their target system, such classical experiments will usually provide the data to compare with. The situation gets interestingly complicated, since the influence of simulation on these experiments is growing. There is a beginning debate on the changing face of experimentation due to computer use (cf. Morrison 2009, 2014; Tal 2013). It is indeed striking to what extent supposedly classical experiments make use of simulation in their experimental setup; examples range from the Large Hadron Collider at Cern to scanning tunnel microscopes.

Our claim is that adjustable parameters play a crucial role in the process of building and applying simulation models. Two interconnected aspects make up our claim: First, both varieties of experiments, or if you prefer another terminology: simulation and classical experiment, cooperate. Second, this cooperation makes use of a feedback loop and works via adjusting parameters.⁴

The outline is the following. In the next section, we start with a brief introduction into mathematical simulation models, their implementation on computers and their application. In addition, equations of state in thermodynamics are introduced, as we will use examples from that area throughout the paper. We chose this field, because it is a theoretically well-founded field of both science and engineering. It provides us with excellent material to illustrate our claims as the parameters of equations of state are of very different nature. They range from the universal gas constant, which can be considered as an adjustable parameter, but one that is found to be valid in a very broad range of situations and closely linked to theory, to mere correlation parameters which are useful only in special situations and epistemologically worthless. We also argue that choosing examples from a theoretically well-founded field provides a stronger argument than choosing them from a field on which little is known and which, hence, has to rely on data-driven models. In the latter field adjustable parameters are important a fortiori. Thus, it is more demanding, and hopefully more fruitful, to show their role in examples taken from the former field.

The topic of parameterization of models has received surprisingly little attention from philosophers. Of course, there is an intense discussion of parameters in the context of curve-fitting and simplicity (see, for instance, Forster and Sober 1994, DeVito 1997, or Kieseppä 1997). There, parameters play the role of degrees of freedom in a quite abstract mathematical sense. In the present paper, however, we assume a physical context in which parameters might have physical meaning. Notable exceptions that discuss this (large) area are those arguing about climate science. Parameterization schemes build a main component in complex climate models (Gramelsberger 2011; Parker 2014) and contain parameters that have to be tuned. The discussion about practices of tuning is just about to start in the climate community (cf. Mauritsen et al. 2012). With thermodynamics, we add a substantially different topic to the applications discussed in this context.

We focus on the development of simulation models and on the decisive role experiments play. Here, experiments include both classical experiments, in which the real world is studied, as well as simulation experiments, in which the implementation of the simulation model on computers is investigated. The importance of the feedback loop in simulation model development is highlighted in Sect. 3, which is based on the comparison of results of computer experiments and classical experiments. This feedback loop is the means by which modeling and experimentation can cooperate closely. Many extant pictures of simulation suggest a “downward” direction (Winsberg 2014) from theoretical model to simulation, or a “bottom-up” direction from phenomena to simulation models (Keller 2003), whereas we underscore that simulation model development is a feedback loop process in which both directions interact.

Section 4 is devoted to a closer look on parameters and feedback and presents the central piece of our argument. We discuss different types of parameters and various situations in which the feedback loop is involved. Typically, simulation models are only simplified representations of their real world targets: parts of the underlying physics may be unknown or so complex that they cannot be incorporated in a tractable simulation model. Thus, workarounds are needed: these often come in the form of models in which parameters are left open – to be adjusted in the feedback loop. One could criticize this from a fundamental standpoint arguing that this is only a poor remedy for a lack of knowledge, and, hence, bane. On the other hand, one can argue that it is boon, because it allows modeling and simulation which otherwise would not be possible. This shows that a critical discussion is needed, to which we want to contribute, namely, by studying the epistemic and practical value of various classes of parameters. We also address the issue of the influence computerization has had on the use of parameters in models. It turns out that it is important and that also in this, there is boon and bane.

Finally, in Sect. 5, we sum up and draw conclusions from the fact that adjustable parameters are boon and bane of simulation. We argue further that simulation modeling adds a new experimentalist twist to mathematical modeling.

2 A Primer on Thermodynamics, Simulation, and Experimentation

Throughout the present paper, we will use examples from thermodynamics to illustrate our arguments. They are chosen from the well-known field of the so-called equations of state. We will only consider equations aiming at describing fluid states (gas as well as liquid) – but not solids. This field lends itself for this purpose as it is fundamental and well known to many scientists and engineers, and it can also be understood by others. Many other areas of science and engineering would have provided equally useful illustrations.

The best known equation of state is that of the ideal gas

$$ p\;\mathit{v}=R\;T $$

(1)

where p is the pressure, v = V/n is the molar volume (volume per mole of substance), and T is the temperature measured in Kelvin. All these quantities are measurable in classical experiments. R is a universal constant (8.314 J mol⁻¹ K⁻¹). Equation (1) has been used before to illustrate issues of philosophy of science, for example, quite recently by Woody (2013) for discussing concepts of explanation. It is known that all substances fulfill Eq. (1) if the density ρ = 1/v is low enough (or the molar volume v is large enough).

The broader concept behind Eq. (1) is that for a given amount of a given substance, p, v, and T are not independent: there is a function which describes the relation between these quantities. Hence the general form of the equation of state is:

$$ \mathrm{f}\left(p,\mathit{v},T\right)=0 $$

(2)

In the low density limit the function f is given by the simple Eq. (1) which is universal in the sense that it holds for all substances. Unfortunately, the relation between p, v, and T is more complicated at higher density and different results are obtained for different substances. The reason for this is simply that at higher densities the interactions between the molecules start playing a role, and hence, the individuality of the molecules matters. We note already here, that while Eq. (1) has substance-wise the widest possible range of application and the “parameter” R has the same value for all substances, there must be ways to tune Eq. (2) to represent a given substance. That tuning is done by adjustable parameters.

Well known examples of such equations are the van der Waals Equation and the Virial Equation of state. In Eq. (2) a pair of independent variables can be chosen (e.g., p and T). The third (dependent) variable (then v) can then be calculated from Equation (2). There is a plethora of proposals for equations describing the p,v,T – behavior of substances for a wide range of conditions. Depending on the form of the function f, their evaluation may only be possible numerically. Furthermore, several solutions may be found and algorithms have to be applied to select the desired one. Different solutions may correspond to different physical states (e.g., gas and liquid) or may be unphysical.

The results from the equation of state can be compared directly to p,v,T – data obtained in laboratory experiments. A good equation of state will describe those experimental data well for a wide range of conditions. But the equation of state can do more. If it describes both gaseous and liquid states, it also describes boiling and condensation. Hence, for example, from Eq. (2) also the so-called vapor pressure curve can be found which describes the dependence of the boiling temperature on the pressure. These results can be compared to experimental data as well. The same holds for results on the heat of vaporization, which can also be obtained from Eq. (2). Calculating these properties, though well-based on general thermodynamics, usually requires algorithms, numerical schemes and a suitable implementation on computers.

It should be noted that mathematics serves as a powerful tool. Once the Eq. (2) is written down together with its parameters, which can easily be done on a piece of paper, a wealth of information on the fluid is specified, like its vapor pressure curve, or caloric quantities. The retrieval of that information can, however, be tedious. In practice, it will depend on the availability of software tools for the evaluation of the given type of equation of state, whether desired results can be obtained with reasonable effort. Although many codes in this field are well tested and considered to be reliable, there is no strict guarantee that the simulation result x ^sim agrees with the (theoretical) model value x ^mod. Let us move from thermodynamics to a general consideration of simulation.

Simulations are based on simulation models. We will assume here that they are given by some set of mathematical equations, which relate input to output. We acknowledge that there are other classes of simulation models, like artificial neural networks, which do not fit into that definition. Their point is exactly to connect input and output in a highly implicit way that is based on extensive parameter adjustments – “learning algorithms” – instead of explicit mathematical equations. The more standard case, where a theory in the form of mathematical equations, thermodynamics in our examples, is at hand, is discussed here. This is the harder case for our argument, because it seems to be less dependent on experimentation and parameter adjustment – but let us see.

In most cases, today, the computer is mandatory to study the model, which is too complex to yield the desired output for a given input in any other way. This is generally true already for our Eq. (2) above. Addressing problems by simulation, hence, connects three important issues: setting up the theoretical model (suitably based on the theory of thermodynamics in our examples) on the one side, and implementing and executing it on computers as well as analyzing the results on the other. The implementation includes steps like discretization, algorithms for solving the equation, coding, and compilation. In practice, there often exist different implementations of the same model on different computers. With the simulation, only the specific model implementation (on the chosen computer) can be studied. There are many situations in which the implementation must inevitably give different results compared to the mathematical model, for example, due to discretization. In other cases differences may simply result from an erroneous implementation. In many cases, the quantity x ^mod – which results from the theoretical model for a given input – is not directly accessible and we can only retrieve numbers for the corresponding result x ^sim of the simulation.

Scientists can vary the model input or other parameters and “observe” how x ^sim changes. This is an experimental activity, but one that does not deal with nature or some system in the laboratory, but rather with the model implemented on a computer, that is, with the simulation model. It is important to note that what is observed in computer experimentation is x ^sim, not x ^mod. However, in many cases, including our case of thermodynamics, one also has a target system, that is, an object of the real world, which is described somehow by the model. Comparison with this system is not only possible, but is an essential part of the simulation activity. Only then, application becomes a topic and a potential problem. The view of simulation presented above then has to be embedded in a wider perspective which includes the real world,⁵ the modeling process and the application of the simulation results (cf. Fig. 1).

Fig. 1

Scheme showing relations between the real world, modeling, simulation, and experiments

We start with a quantity x ^real in the real world we want to model. The corresponding entity in the (theoretical) model is x ^mod. As the model is too complex to be evaluated directly, it is implemented on a computer and simulations are carried out as described above. These simulations yield a quantity x ^sim which can eventually be compared to results of experimental studies of the real world x ^exp. In general, we cannot know x ^real nor can we know x ^mod, we can only compare the corresponding properties of x ^exp and x ^sim. There are two types of experiment in play. One from “below” that provides measured values, the other from “above” that provides simulated values.⁶ Up to this point, the picture coincides with prominent schemes of modeling, like R.I.G. Hughes’ DDI account (1997), or Reichenbach’s (1964, 102/103) appreciation how mathematical deduction (on the model level) and observation work together in science.

The discussion around Eq. (2) presented above highlights the role theory plays in this context and reminds us not to interpret Fig. 1 too literally: Eq. (2) describes a priori only p,v,T properties. But based on arguments which combine some very basic physical statements with mathematics, it can be shown that it describes also properties which are at first glance unrelated to p,v,T properties, like boiling conditions, caloric properties, and many more. It is the success of such predictions that convinces students of taking the effort of studying the theory which enables them.

Up to now, simulation was a means for revealing what the model says about the property x under investigation. This is regularly a task for which there is no alternative to using a computer. Still, the basic rationale is the standard one: The analysis and evaluation of the theoretical model via comparison to the target system.

In general, the quality of a model depends on two aspects that counteract each other. It depends both on adequacy of representation, else the model would not yield results revealing anything about the target system, and tractability, which is prerequisite for obtaining some result at all. Here is where computers have changed the picture. They can handle very long and convoluted iterative algorithms that would be intractable for human beings and, hence, make models tractable which otherwise would be useless.

Figure 1 is rich enough to account for our illustrative case. Equations of state (Eq. 2) have parameters which need to be adjusted to some data on the fluid that they are meant to describe. That data is usually taken from laboratory experiments. An alternative is results from computer experiments. In most cases, for that purpose molecular simulations based on force field are used in which the latter describes the interactions between the molecules. In the molecular simulations, the p,v,T – behavior or other macroscopic thermodynamic properties can be studied based on a model of the interactions between the molecules. The results of these simulations always fulfill Eq. (1) in the low density limit where the molecules are so far apart that the interactions play no role. But at higher densities, when the interactions start playing a role, they deviate from Eq. (1) but open the door to formulating equations of the type (2) which depend on the interaction model. Thus, the computer experiments yield new opportunities compared to classical experiments. The type and strength of intermolecular interactions can be systematically varied and the effect of that variation on the p,v,T – behavior can be studied. This is widely used in developing new mathematical forms of equations of state (see, e.g., Wei and Sadus 2000).

3 Simulation Model Development as Feedback Loop Process

For our claim about the significance of adjustable parameters, we need a more complex picture of experimentation. In this section we highlight a particular feature of simulation model development, namely, a feedback loop of model adaptation. It is depicted in Fig. 2, which derives from Fig. 1 by adding one arrow that closes the modeling loop. It is basically a classical feedback control loop which aims at minimizing the differences between a variable (here: x ^sim) and a set value (here: x ^exp). The two quantities which are compared need not be scalar quantities but may have many entries or be, for example, trajectories over time. There are also many ways of carrying out the comparison.

Fig. 2

Same as Fig. 1, but one arrow added, pointing to the left and closing the feedback loop of modeling

This feedback loop easily appears as marginal, as a pragmatic handle for fine-tuning and correcting imperfections of the (theoretical and simulation) models. We argue that it is not. Adjusting parameters fulfills essential functions in simulation modeling. Repeated comparison of the two types of experiment guides the modeling process. During this phase, the process consists in adjusting parameters. The model is explored via (simulation) experiments, motivated by comparison with (classic) experiments. We have hence a cooperation of both types of experiments that is the nucleus of model development via adjusting parameters. However, the cooperation gets even more intertwined when one takes into account that the measured quantities themselves might be partly determined with the help of simulation.

Basically two sorts of actions can be taken in modeling when the comparison of the simulation results and the experiments do not yield the desired results: a) the model structure can be adapted, that is, the set of equations is modified, for instance, to change a description or include a previously neglected effect, or b) model parameters are changed. It is this second option that we are interested in here. Parameterization schemes can be considered as a sort of auxiliary constructions that are intentionally used for dealing with missing knowledge and the inaccuracies of existing knowledge. The simulation model is designed, so that it contains parameters that can be adjusted in the course of the further development.

The remainder of the paper focuses on the role of adjustable parameters. The reasons for using adjustable parameters are discussed in more detail, and it is shown that adjustable parameters form an essential component of simulation modeling. While models with adjustable parameters have been around much longer than computers, practical hurdles had limited their use in the past. The easy availability of computers and optimization software has tremendously lowered these hurdles. It has become much easier to utilize the adaptability of models, so that it has become much more tempting to succumb to the lure of making models fit by adjusting enough parameters.

Parameter adjustment is only one way of model adjustment. Besides adjusting the model parameters, the structure of the model (the mathematical equations) can be adjusted to obtain a better fit to experimental data. The latter procedure is very closely related to parameter adjustment if the equations are changed without any other physical reasoning than obtaining a better representation of some data. The equations themselves then are seen as a sort of parameters that can be adjusted.⁷ We will also address this issue in the present work.

The equations of state, which we use as examples here, contain adjustable parameters which are usually determined from experimental data. The exception is the equation of state of the ideal gas (Eq. (1)). But even in that case, it can be argued that R was once not more than an adjustable parameter. But it is no longer, R has turned out to be a universal constant. It is beyond the scope of the present work to discuss the far reaching consequences of that universality which include the definition of temperature and the atomistic nature of matter. By the way, these relations provide again an excellent example for the success of the combination of mathematical deduction with observation. We will rather focus on the consequences of the adjustment of parameters in equations of state and the role the computer plays in this.

As simple examples, we use the van der Waals equation:

$$ p=\frac{R\;T}{\mathit{v}-b}-\frac{a}{\mathit{v}^2} $$

(3)

and the Virial equation of state in the following form:

$$ \frac{p\;\mathit{v}}{R\;T}=1+B\frac{1}{\mathit{v}}+C\frac{1}{\mathit{v}^2} $$

(4)

The researchers who have introduced these equations, J.D. van der Waals and H. Kammerlingh Onnes, received Nobel prizes in 1910 and 1913. These equations, though both with strong foundations in physics and mathematics, contain adjustable parameters, namely, a and b in Eq. (3) and B and C in Eq. (4). These parameters are needed to account for the individuality of different fluids, that is, water is different from nitrogen. The parameters are not necessarily simple numbers but can be functions of variables. For example, the theory behind Eq. (4) yields that B and C are functions of the temperature, but not of pressure. In the original version of Eq. (3) a and b were numbers. However, in later versions of Eq. (3), a was considered to be a function of temperature. Adjusting functions is obviously more flexible than adjusting numbers.

4 Adjusting Model Parameters: A Closer Look

Speaking about adjustment of parameters invokes a field of similar terms with (only) slightly differing connotations. Calibration, for instance, is used in the context of measuring instruments. Hence, using calibration of parameters makes models look a bit like precision instruments. Tuning, on the other side, has a slightly pejorative meaning, though it is used in some areas of science as the standard term. Anyway, we chose adjusting because it seems to be neutral and does not appear to be a good or bad thing from the start – though we do not claim our terminology is without alternative. Adaptation, for instance, has an equally neutral flavor.

In this section, we discuss a spectrum of situations in which parameters get adjusted. We adopt here a simple scheme of a simulation model, which is based on systems theory (cf. Fig. 3). The process model aims at describing a certain set of quantities y, which we will call output variables here. The output depends on the input, which is described by another set of quantities u, the input variables. Both y and u belong to the quantities which occur in the model. The latter will be called model variables x here. The set of the model variables x may contain quantities which are neither input nor output (i.e., internal variables). The question which subset of x is considered as input and which as output may depend on the application. In the models in which we are interested here, y and u describe properties of the target system. Ideally, y is a measurable quantity and u can be set in experiments.

Fig. 3

Parameter adjustment in models

4.1 Model Parameters

Besides the input variables u, many models require the specification of model parameters p. These do not necessarily correspond to anything in the real world. Cleverly setting the model parameters allows improving the quality of the model regarding its output y. The parameter adjustment involves some kind of optimization procedure. The goal of the optimization is to improve the agreement of the model output y with some reference data, usually experimental data y ^exp (cf. Fig. 3). We do not presuppose some elaborated formal algorithm for optimization. A simple trial-and-error method is eligible for “method,” too. We should point out, however, that mathematical optimization methods reach far beyond what can be handled by simple trial-and-error. Such methods often act like black boxes for thermodynamics modelers. We leave the detailed consideration of the optimization part for another paper.

For example, in the van der Waals equation (3), the input variables may be chosen to be the temperature T and the molar volume v, and one may be interested in the result for the pressure p at those chosen conditions. The calculated result will depend on the choices made for the parameters a and b. Obviously, if some p,v,T data points are available for a given substance, the parameters a and b can be adjusted to these data. Thus, the results obtained for a and b will depend on the choice of the data set to which they are fitted, and also on the way they are fitted. For parameterizing equations of state different types of data are used (e.g., besides p,v,T data also data on vapor pressures, data on the critical point of the fluid, or caloric data). The calculation of such properties regularly involves numerical procedures and as a consequence computers are needed. This becomes especially important in the parameterization which is an optimization task that regularly involves a large number of evaluations of each property. Computers enable adjustments which were not feasible before.

On the one side, adjusting model parameters is obviously a boon, as it can make models work. In many cases it is the key to making them work. Even an otherwise poor model could be augmented by a suitable parameter fit so that it gives fair representations of y. In a community which is used to judging models solely by their ability to describe certain properties y, this is clearly attractive.

In the case of equations of state, there is an obvious need for an adjustment of parameters. With only a few exceptions, we are not yet capable of predicting properties of real fluids from first principles. Hence, models describing such properties must be trained by some experimental data. The way to do this is by adjusting model parameters. In the field of fluids, the predictions from first principles are presently basically limited to calculating ideal gas properties from Schrödinger’s equation. But equations like (3) and (4) are far more than some mathematical form which is fitted to data. We mention only some examples:

• (a) by virtue of their derivation they contain Eq. (1) as limiting case,

• (b) the B parameter of Eq. (4) can directly be related to intermolecular pair interactions and was for a long time the most important source for quantitative data on them,

• (c) the simple Eq. (2) predicts the existence of phenomena like critical points or metastability of fluid phases and relates them to other fluid properties in a consistent way.

These examples highlight the unifying power of the thermodynamic theory and are examples for the benefits of combining theory and experiment.

On the other side, the adjustment is a bane, since it does not remove flaws of models; it rather disguises them. Even an obviously wrong model, that is, one with internal logical contradictions, can give fair representations of y after a suitable adjustment of parameters. For a scientist, who is interested in obtaining insights from models, this is scary. He may be inclined to discard models as worthless if they only work after adjusting parameters to data which the model aims to describe. What is the use of a description of properties of something which can only be established based on the knowledge of the same properties?

This point is illustrated again using equations of state as an example. These equations can be used for describing mixtures. The key to this is finding expressions for the parameters of the equation (like a and b in Eq. (2)) which hold for the mixture. These mixture parameters are usually calculated from the corresponding pure component parameters and the composition of the mixture via so-called mixing rules. With the exception of the mixing rules for the parameters of Eq. (3), which can be rigorously determined from the principles of statistical thermodynamics, these mixing rules are empirical. They contain parameters which usually have to be adjusted to mixture data. Nevertheless, they can be submitted to some tests which can be of logical nature (i.e., if a pure component is formally split up in two identical components, the pure component result should be obtained also from the mixture model) or based on fundamental findings of thermodynamics like those from statistical thermodynamics mentioned above. It is known that mixing rules which fail both in the logical tests and those from statistical thermodynamics, can nevertheless turn out to work well in practice, if the parameters are suitably adjusted. For examples, see Mathias et al. (1991).

4.2 Proliferation of Variants

Moreover, adjusting parameters leads to what we call a “flood of flavors,” because the results for the parameters and hence all results obtained with the model will depend on the choice of the data set to which the parameters are fitted, and on the way they are fitted. The flood may turn into a deluge if also variants of the mathematical form are included, which are only introduced to improve some fits and have no other basis.

In fact, computers have opened the gates to that flooding. The possibility to easily create and check variants of some model on empirical grounds is at first sight positive. Upon closer inspection, the picture changes. Firstly, the plethora of variants of a given model will rarely have epistemic value. But even from an entirely instrumental standpoint, it may be detrimental. A plethora of versions of a model will create an obstacle for anybody who wants to use the model. Which one to choose? By facilitating the creation of sprawling mutations of models, computers have contributed to the fragmentation of research.

Let us only consider the van der Waals equation, Eq. (3), as an example. It was developed in 1873. Meanwhile there are more than 400 equations of state (so-called cubic equations of state) which can be considered to be variants of that single equation (Valderama 2003). While this gives, of course, enormous credit to the ground-breaking work of van der Waals, it is also distressing. The variants can hardly be classified on theoretical grounds. Rather, historical (when were they developed?), sociological (how well are they received?) or pragmatic arguments (what practical benefits are offered?) and classifications are used. There are some very successful variants which are widely used, and there are certainly elder versions which have technical drawbacks, but there is a plethora of variants that are very similar. Many of these have been used only by the group which has proposed the equation. This danger has nicely been captured by D. Frenkel in his paper on the “dark side” of simulations: “In the past, we had to think about the role of simulations because they were expensive, now we have to think because they are (mostly) cheap” (2013).

Note that the above discussion only addresses the number of mathematical forms of the equation. For each of these there exists a plethora of specific variants. For example, for describing mixtures, one can combine a given equation of state with many different mixing rules. Due to the combinatorial explosion, most of the options have never been explored and never will. And there is very likely no loss in not doing so. Furthermore, we have not addressed that, even in the simpler case of a single pure component, there is a practically unlimited number of options for choosing the data set used for the parameterization – each of which will yield a different set of model parameters.

This sprawling of variants cannot be solely attributed to the use of computers but it is certain that computers have strongly accelerated that development. They have also favored the increase of the number of parameters in a model of a given object. While the van der Waals equation (Eq. 3) only has the two parameters a and b, modern equations of state may have 30 or more adjustable parameters.⁸

4.3 Necessity of Adjusting Model Parameters

What are the reasons that make this parameterization problem so endemic and in a sense unavoidable? In general, any mathematical model presents an idealized version of the real world target system. There is always more abundance in the target system than in some mathematical model equations. Mathematics can be seen as a science which works with objects that are so simple that they can be fully described – which is never possible for an object of the real world.⁹ Hence, there may be unknown properties of the target system that should be included in the model, but are not. Leaving open some model parameters and adjusting them to experimental data can be considered as a pragmatic remedy for this.

Even if all properties of the target system which have an influence are known, it can still be prohibitive to explicitly account for their influence in the model. There may simply be a lack of theories, or existing theories might be so complex that they would make the model intractable. Adjustable parameters are of prime importance in this context. They enable using simplified but tractable models. Such models may only be loosely related to the real object and may be obvious over-simplifications. But leaving open some parameters in such models and adjusting them in a clever way can make them work. This is at the core of engineering. Engineers look for simple models which will “do the job,” that is, produce good results in a certain application. Their main interest is in the application, not in the physical interpretation. Carefully parameterized simple models can give astonishingly reliable and useful results. As the parameterization involves some comparison with experimental data, it even guarantees a certain quality of the model (at least for the representation of the data that was used for the parameterization). All this is relevant not only in engineering but in many parts of science. Furthermore, accuracy matters and for the reasons mentioned above even good models will never be perfect. Parameterization can be used for alleviating this too.

For example, in the van der Waals equation (1), the parameters a and b have a physical meaning. They are associated with attractive (a) and repulsive (b) interactions between the particles. It is well known that there are many different types of attractive forces, which are all lumped into the a parameter. It can, hence, be considered as an “effective” parameter. Such parameters are meant to describe the influence of a certain class of physical phenomena within a given model. In addition, the parameter b can be considered as such an effective parameter describing repulsion. Despite the crude simplifications in the assumptions on the intermolecular interactions, the van der Waals equation and its mutants have been extremely successful in describing real fluids. There are two main reasons for this. The first is that the structure of the equation (which comes from theory) is able to reproduce qualitatively the most important features of the behavior of fluids like the coexistence of vapor and liquid at certain conditions, the ideal gas limiting behavior etc. The second reason is that the equation contains the parameters, which can be suitably adjusted. Both reasons act together.

Above, we have discussed how simulation takes advantage of the possibility to iterate the feedback loop. It is the very point of the feedback modeling loop that the model is adapted to yield some global behavior. Consequently, the parameters which are used to achieve this do not necessarily follow their physical interpretation – and they do not even need to have such an interpretation at all.

4.4 Parameters with and Without Independent Physical Meaning

In principle, any variable in a model can be used as adjustable parameter. Two cases should be distinguished, depending on the question whether the parameter has an independent physical meaning or not. Independent physical meaning is used here in the sense that there is a physical interpretation outside of the context of the parameter fitting. For reasons of illustration, consider a model for describing the dependence of a physical property y (output) on some other physical property u (input). In the model, it is simply assumed, without any physical argument, that the relation is linear, hence:

$$ y=a\;u+b $$

(5)

where a and b are adjustable parameters. The parameter a describes the sensitivity of y on changes of u, which is a physical property in the example. Whether the parameter b has a physical interpretation depends on the range that the input values for u have. If u = 0 is a physically reasonable scenario, then b is simply the result for y for that case. All this is trivial and not the case we want to make, because here the physical interpretation is no more than some curve discussion in high school mathematics. The case we are interested in is when the linear relation of Eq. (5) is resulting from some physical theory; and there could be a possibility for calculating a from properties that are not y and u. Still, a could be used as adjustable parameter in the fit using data on y and u.

We use the van der Waals equation for a further illustration of the above: assume its parameters are fitted to experimental p,v,T - data of some liquid. On closer inspection of Eq. (2), one finds that the liquid density at high pressures is determined by the b parameter. Hence, one can physically interpret the b parameter as describing the liquid density at high pressures. This is considered here as an interpretation in the context of the parameter fitting, and hence not an independent physical interpretation. However, as stated above, by virtue of the derivation of the van der Waals equation, the b parameter has a deeper meaning. It describes repulsive intermolecular interactions. These obviously become very important in liquids at high pressures, where the distances between the particles in the fluid become very low. Repulsive interactions can in principle also be determined independently, namely, from quantum chemistry. Unfortunately, the derivation of the van der Waals equation is based on such crude simplifications that there is no way to relate or predict the b parameter from independent sources of information, like quantum chemistry.

The above shows different things: while it is fair to say that b is related to repulsive interactions, there is no way to establish such a correlation quantitatively. An important consequence of this is that the numbers for b obtained from fitting should not be over-interpreted as carrying useful quantitative information on the repulsive interactions. That this is not possible becomes also evident when considering that the numbers obtained for the b parameter of a given real fluid will depend strongly on the choice of the data set used for the fit. Nevertheless, it is obviously a merit of the van der Waals equation that it gives structural insight into the importance of certain interactions at certain conditions, in our example the repulsive interactions in liquids at high pressures. We now return from the example to our main line of argumentation.

First, consider the case where the variable, which is used as parameter, has an independent physical meaning. By using it as adjustable parameter that physical meaning is given up in the first place. A number is assigned to that variable in the feedback loop based on pragmatic considerations of the overall model quality, and disregarding the physical interpretation that the resulting number may have. However, one may try to recover the physical interpretation after the parameterization by comparing the result with some independent information on the property, if such information is available. The result of the comparison may well be disastrous without compromising the usefulness of the overall model. But such an outcome will shed a bad light on the explanatory power of the model. On the other hand, it might turn out that the fit has produced a number which is “physically reasonable,” that is, which meets some expectations based from considerations that were not included in the fit. This would be a clear sign of the epistemic value of the model, even in a strong sense where it not only predicts physical phenomena qualitatively but also quantitatively.

If independent information on a variable (parameter) is available, one may ask why that independent information was not used right away in the model. A good answer to that question would be the lack of accuracy of the information. If the output of a model strongly depends on a variable which cannot be measured accurately, the variable cannot be used as input variable straightforwardly. In such a situation, the procedure which we have called parameter fit here could be a part of a scheme for data estimation. Pushing this point further, the use of physical variables as adjustable parameters can be considered as a part of a measuring scheme for the associated properties which involves both classical experiments, modeling and simulation.¹⁰

Let us now turn to the second case where the variable which is used as parameter has no independent physical meaning. At first glance, that case may seem to be trivial. One simply obtains some numbers from the fit and there is no need nor possibility to interpret the results for these numbers. All there is to do is to check the overall model quality. Maybe some kind of curve discussion of the fit can be added (cf. the example around Eq. (5)).

Things become more interesting if we consider why adjustable parameters without physical meaning are used. Basically, this results from operative requirements in the modeling process in which highly complex target objects have to be described with models that are still feasible. This may make it attractive to represent a complex real situation by a model which was initially developed for describing a much simpler physical situation. These models have parameters which describe physical quantities in the simple context for which they were initially developed. They therefore also carry names of physical properties. But in the complex context in which they are used as fitting parameters, the original physical meaning is lost, they degenerate to empirical parameters. However, this is disguised by the fact that they carry names of physical quantities. This has caused much confusion. One should refrain from physical interpretations of results of fits of such parameters, despite the fact that they often carry names of physical variables. An example might clarify the point.

For instance, for modeling liquid mixtures, there is an entire class of models, the so-called lattice models, in which the liquid is represented by particles at fixed positions on a lattice. In the simplest version, each particle occupies one lattice site. This picture is taken from an ideal crystal. For describing liquids, in which molecules are constantly moving around, changing their neighbors, this is a bold simplification. The key parameters in such models are those describing the interaction energies of neighbors on the lattice. For example, in a mixture of two components A and B, there are three such energies, those of the AA, the BB, and the AB interaction. If such simple models were applied to describe some crystal solid, one could hope to interpret the results for the interaction energies determined from a fit to some data in a physical way and compare them to independent data. In the context of modeling liquids, there is no hope in such an endeavor. The numbers obtained for the interaction energies from fits to liquid mixture data have no meaning, even though they are still called interaction energies.

On the other hand, parameters of models which were chosen for entirely mathematical reasons (e.g., coefficients of Taylor series expansions) may turn out to have a strong independent physical meaning. For example, Eq. (3) can be considered as a Taylor series expansion around the state of the ideal gas and B and C are just the first two coefficients of that expansion. The theory of statistical mechanics shows that these coefficients are directly related to the energy of pair interactions in the gas.

In developing computer models, also hybrid schemes are applied in which model parts with a strong physical background and parameters that have an independent physical meaning are combined with empirical parts. Here, the parameters are merely there to improve some model results, which without the use of these parameters would be inacceptable.

Again, the van der Waals equation provides an example. Originally, the parameter a was a number with a certain value for each fluid. But it was soon realized that for accurate descriptions of the thermodynamic behavior over a large temperature range, namely, of the vapor pressure curve, substantial improvements could be achieved by allowing for a temperature dependence of a. The mathematical forms for describing a(T) are empirical and so are the adjustable parameters in these forms.

4.5 Parameters in the Implementation

So far, we have only discussed model parameters. We have neglected the fact that the (theoretical) models often cannot be studied directly. They first have to be implemented on computers. Different implementations of the same model will usually not yield exactly the same results. As a consequence, the implementation, which is a part of the feedback loop of modeling, will influence the model parameterization. Aside from implementation errors, the differences between different implementations of one model are luckily often small enough to be neglected. Model parameters determined in one study are regularly and successfully used in other studies, even though the model implementations differ. However, there is no guarantee that this is the case.

There is more concern about parameters which occur in the model implementation. Prominent examples of such parameters are those used in the discretization of models or those used to control numerical solvers. Ideally, these parameters are chosen from ranges in which the influence of the parameter on the simulation result is negligible (e.g., the grids used for the discretization must be “fine enough”). But it may be very difficult to guarantee this.

When there is an influence of such parameters on the simulation results, they can actively be adjusted in the modeling feedback loop to improve the simulation results. This is much more problematic than adjusting model parameters, as it is implementation-dependent. It forecloses the discussion of the model without referring to the specific implementation. It also may be misused to feint a success of the model, which cannot be attributed to the model but just to a deliberately tuned implementation.

We think that adjusting parameters of the implementation should always be done based on the consideration of minimizing the influence of those parameters at acceptable simulation effort. It must never be used for tuning simulation results in the modeling feedback loop.¹¹

4.6 Models and Correlations

Both the term “model” and the term “correlation” are used for referring to descriptions of objects of the real world. Model has a better reputation than its counterpart correlation. Correlations are often considered as “some kind of (empirical) model,” but one where physical theory is not invoked. Rather, statistical considerations play the leading role – largely independent from the physical properties of the particular target system under investigation.

In the framework that we have presented here, a correlation is just an extreme version of a model. In Fig. 3, the term model could be replaced by correlation and nothing else would have to be changed. The feedback loop is even more essential for the correlation than it is for the model. This is due to the fact that the correlation relies on adjustable parameters, either fully or at least in essential parts. It does not even have to have any physical background. A correlation can, for example, just be a mathematical form, which is taught to describe a physical situation by adjusting its parameters. Artificial neural networks provide a telling example, since they work with a generic structure, while the parameter adjustments determine the behavior nearly completely. It is noted here that most physical phenomena can be described by suitably parameterized mathematical expansions around a given point (Taylor series, Fourier series etc.).

From this we conclude that there is a continuous transition between physical models and correlations, depending on the degree in which they rely on adjustable parameters and whether the parameters of correlations are open to a physical interpretation as in the van der Waals equation (see 4.4. above). The number of adjustable parameters in a physical model should never exceed the number of adjustable parameters in a purely mathematical (statistical) correlation of the studied phenomenon, which yields a similar quality of the description. Else the physical theory does supposedly not work.

5 Conclusion: Boon and Bane

The cooperation of experiments, that is, of simulation and classical experiments, plays a crucial role in simulation modeling. This cooperation thrives on the feedback loop in modeling which provides the basis for adjusting parameters. Notably, we monitored the significance of parameter adjustments even in the area of thermodynamics and equations of state where theory is highly developed and well-grounded. Adjustable parameters will arguably not be less significant in fields with less support from theory.

Adjusting parameters is often the clue that makes a model work, it is a boon. Using the term applicability in an engineering sense, which can broadly be identified with usefulness for solving practical problems, it is fair to say that adjusting parameters is often the prerequisite for the applicability of a model. At the same time, the adjustment of parameters limits the applicability of models. The model will often only be useful for describing scenarios which are not “too far away” from the scenarios that were used for the fit. We cannot enter into the interesting discussion of this in detail here and just mention that the question how far a model carries beyond the range where it was parameterized is closely related to the quality of the theory behind it, that it is by no means trivial to establish metrics to measure what “far” means, and that the answers to the latter question will be strongly case-dependent. In any case, it must be clear that the adjustment of parameters can simply not replace a sound theory.

Let us consider the equation of state of the ideal gas (Eq. 1) as an example. We start by simply considering it as a model to describe p,v,T data of gases at low densities, and state that the region of the applicability of the model is extremely large as it holds for all substances. The model allows far-reaching predictions as the “parameter” R does not depend on the substance. Equation (1), which is often also called “Ideal Gas Law” would obviously lend itself to a closer discussion of the relation of the terms model and theory, but we must refrain from this here. We rather move to other equations of state, for which the picture changes. Let us use the van der Waals equation (Eq. (3)) as an example and assume first that we are merely interested in using it for describing properties of a certain pure fluid. For this, we need to have numbers for the parameters a and b of that fluid. They must be obtained from an adjustment to some experimental data. Once this is done, we can make all sorts of predictions using Eq. (3) but the quality of these predictions will strongly depend on the relation between the data that were used for the fit and the data which are to be predicted. Interestingly, there are common notions in the thermodynamic community as to which data are to be used for parameterizing equations of state for fluids. For example, as a rule, experimental data on the so-called critical point are used, if they are available. The reason is that parameterizations based on such data are found to be more broadly applicable than competing parameterizations. To summarize, the following issues are inextricably entwined: the model with its parameters, the way the parameters are determined, and the applicability of the model in certain situations. Taking this into account, a scheme for comparing the quality of different models would be to use the same data for the parameterization, to apply the models for studying the same quantities, and to compare the quality of the results.

The limitation of the applicability of a model by its parameterization is not a bane in itself. But it becomes a bane when it is overlooked. We must get used to never think of models without considering the way they were parameterized.

In a sense, adjusting parameters is strongly guided by predictive quality over a certain – and maybe very limited – range. While enabling application, this procedure diminishes the explanatory capacity of simulations, because the iterated adjustments tend to obscure or convolute what can be attributed to general theory, parameterization strategy, and particular parameter adjustment. In brief, adjustable parameters are boon and bane of simulation. Empirical, theoretical, and computational aspects complement each other.

Though our results belong to the philosophy of simulation, they point toward a more general philosophical perspective. Let us take simulation in terms of mathematical modeling. Simulation then does not merely extend mathematical modeling, but adds a new twist to it. Now classical and simulation experiments cooperate, building on the feedback loop and on adjustable parameters. Our investigation thus adds a new twist to the so-called “new experimentalism” of Hacking, Franklin and others. They highlighted the importance of experimental (instead of solely theoretical) knowledge in the sciences. Philosophy of science would then examine how experimental knowledge actually arrived at and how this knowledge functions. The rationality of science then is not distilled from some logic, but from actual scientific practices. Our paper contributes to this line of reasoning and extends to practices of simulation. The interesting twist then introduces an empiricism that is different from earlier accounts of modeling in an important way. It neither explores the theoretical model, nor inserts measured values (as in semi-empirical methods) for parameters hard to calculate. The cooperation of experiments and the exploratory/ empiricist nature of adjusting parameters extend well into the home territory of theory.