Overview of Computational Modeling and Simulation

Abstract

Scientific research involves the formulation of theory to explain observed phenomena and using experimentation to test and evolve these theories. Over the past two decades, computational modeling and simulation (M&S) has become accepted as the third leg of scientific research because it provides additional insights that often are impractical or impossible to acquire using theoretical and experimental analysis alone. The purpose of this chapter is to explore how M&S is used in system-level healthcare research and to present some practical guidelines for its use. Two modeling approaches commonly used in healthcare research, system dynamics models and agent-based models, are presented and their applications in healthcare research are described. The three simulation paradigms, Monte Carlo simulation, continuous simulation, and discrete event simulation, are defined and the conditions for their use are stated. An epidemiology case study is presented to illustrate the use of M&S in the research process.

FormalPara Overview

Scientific research involves the formulation of theory to explain observed phenomena and using experimentation to test and evolve these theories. Over the past two decades, computational modeling and simulation (M&S) has become accepted as the third leg of scientific research because it provides additional insights that often are impractical or impossible to acquire using theoretical and experimental analysis alone. The purpose of this chapter is to explore how M&S is used in system-level healthcare research and to present some practical guidelines for its use. Two modeling approaches commonly used in healthcare research, system dynamics models and agent-based models, are presented and their applications in healthcare research are described. The three simulation paradigms, Monte Carlo simulation, continuous simulation, and discrete event simulation, are defined and the conditions for their use are stated. An epidemiology case study is presented to illustrate the use of M&S in the research process.

FormalPara Practice Points

• There are three main simulation paradigms: Monte Carlo simulation, continuous simulation, and discrete event simulation, however a hybrid simulation combining any two paradigms is also possible.

• The Monte Carlo simulation paradigm refers to the methodology used to simulate static, stochastic system models in which system behavior is represented using probability.

• The continuous simulation paradigm refers to the methodology used to simulate dynamic, continuous-state, time-driven system models.

• The discrete event simulation paradigm refers to the methodology used to simulate dynamic, discrete-state, event-driven system models, such as a queuing model.

• Modeling methods include system dynamics models and agent-based models; both methods frequently are used for complex healthcare and medical systems, including epidemiological applications surveyed here.

Introduction

Modeling and simulation (M&S) long has been used for education and training in the healthcare domain. Most medical practitioners are familiar with the use of visual models and simulations and simulation-based instructional applications to enhance the transfer and acquisition of knowledge. They also are familiar with the use of task trainers, medical mannequins, and immersive interactive virtual reality for training where the objective is to control performance variability (i.e., minimize error) by improving trainee reliability. However, the use of M&S as a computational approach to support and enhance healthcare research is a more recent and perhaps less familiar topic for medical practitioners. The focus of this chapter is to explain how M&S is used in system-level healthcare research and to present some practical guidelines for its use.

Computational modeling and simulation (M&S) refers to the use of models and simulations, along with the associated analysis, visualization, and verification/validation techniques, to conduct a simulation study. The subject of a simulation study is usually described as a system. A system is a combination of components that act together to perform a function not possible with any of the individual components. A system that is the subject of a simulation study is called the simuland . A model is a mathematical or logical representation the simuland. Selection of a model must consider both the relevant features of the simuland and the questions about the simuland that are to be addressed. A simulation is a process for executing a model. Selection of a simulation methodology depends on the mathematical characteristics of the model.

Historically, M&S has been viewed as an important research tool in numerous disciplines or application domains. Research in most domains often proceeds through a sequence of phases that include understanding, prediction, and control [1]. The initial phase is used to gain an understanding of how events or objects are related. An understanding of relationships among objects or events then allows the modeler to begin making predictions and ultimately to identify causal mechanisms. Finally, knowledge of causality enables the user to exert control over events and objects. Research moves from basic to more applied levels as progression is made through these phases. For example, the Human Genome project was undertaken to understand the complete sequencing of chromosomal DNA in human beings. Knowledge of the human genome helps to make predictions regarding genetic variation and can lead to more reliable diagnostic tests and medical treatments applied at the genetic or molecular levels.

M&S is closely linked to all phases of research. At the more basic levels, research is guided heavily by theory. Models are often used to represent specific instances of theories, to differentiate between competing theories, or to exhibit underlying assumptions. Likewise, simulations are used to test predictions under a variety of conditions or to validate theories against actual conditions. At the applied levels, simulations also are used to control events and objects. Simulations in the form of mock-ups or prototypes are used in the creation of products and systems to validate predictions regarding operational requirements, specifications, and user/customer satisfaction.

Although this description of the research process admittedly is simplistic, it does underscore three important points regarding M&S. First, M&S is intimately related to all phases of the research process. M&S is used to generate and refine the theories that help us understand our world as well as the technology we use to interact with the world. Second, the description is generic and highlights where M&S can be applied in any domain where individuals are engaged in research. Thus, biologists, chemists, sociologists, economists, and historians all can use M&S to help formulate research questions, conduct experiments, evaluate theories, and add to their respective bodies of knowledge. Third, the description also shows the different aspects of M&S emphasized along the basic/applied research continuum. Thus, at the basic end, M&S is used more as a research tool whereas at the applied end, it is used either to create products or even may be a product in and of itself.

The remainder of this chapter is organized in four sections. In the first section, Simulation Methodologies, we focus on simulation paradigms. The three simulation paradigms are defined in terms of the system classifications associated with the simulation model. In the second section, Selected Modeling Methods, we describe two modeling approaches often used in healthcare research, system dynamics models and agent-based models. An example of applying M&S to healthcare research is presented in the third section, Example Healthcare Applications. An epidemiology problem is investigated using different modeling approaches and simulation methods to illustrate some of the practical issues that must be considered. In the fourth section, Conclusion, several challenges associated with applying M&S in healthcare research are identified and briefly discussed.

Simulation Methodologies

In this section, we identify the three simulation paradigms, Monte Carlo simulation , continuous simulation , and discrete event simulation , and discuss the process for selecting an appropriate paradigm. Selection of a simulation paradigm depends primarily on the characteristics of the model that is to be simulated. Model characteristics are defined in terms of the mathematical properties of the functional representation for the model. Each simulation paradigm is designed for use with models having a specific combination of these system characteristics. A fourth simulation methodology, hybrid simulation, refers to simulation methodologies that consist of utilizing two or more simulation paradigms to simulate a single simuland model.

System Characteristics

A model often is represented mathematically using the definition of a function [2]. A function is a mathematical construct consisting of three components, the domain set X, the codomain set Y, and the rule of correspondence Γ. The domain set consists of the set of system inputs x(t) ∈ X, the codomain set consists of the set of system outputs y(t) ∈ Y, and the rule of correspondence consists of the mapping of inputs to outputs denoted as Γ : X → Y or Γ{x(t)} = y(t). The system state at time t₀, q(t₀), is the (minimal) information about the system at t₀ such that the output of the system for t ≥ t₀ is uniquely determined from this information and the system input for t ≥ t₀. The state space Q of a system is the set of all possible values that the state may take.

System characteristics are defined as all inclusive, mutually exclusive descriptor pairs that are based on the mathematical properties of the model functional representation. Definitions for these descriptor pairs are presented in the following.

• Static or Dynamic – A system is said to be static if the system output at time t_i is dependent only on the system input at time t_i. A system is said to be dynamic if the system output at time t_i depends on the system input for t ≤ t_i. Dynamic systems are called systems with memory while static systems are called systems without memory. The output of a static system at time t depends only on the input to the system at time t. The output of a dynamic system at time t depends on both the input to system and the state of the system at time t.

• Deterministic or Stochastic – A deterministic system is a system in which all system outputs are deterministic. A stochastic system is a system in which one or more system outputs have uncertainty or variability. In this case, the system output is characterized as a random process and a probabilistic framework is required to describe system behavior.

• Continuous-State or Discrete-State – A continuous-state system is a system in which the state space Q consists of elements q(t) that assume a continuum of real values; that is, q(t) ∈ R (real numbers). Examples of continuous-state systems include many physics-based systems where system variables (position, velocity, magnitude) have real number values. A discrete-state system is a system in which the state space Q consists of elements q(t) that assume only discrete values; that is, q(t) ∈ I (integer numbers). Examples of discrete-state systems include many service systems where system variables (people counts, resource counts, part counts) have integer number values.

• Event-Driven or Time-Driven – In discrete-state systems, state changes occur only at distinct instants of time as variable values change instantaneously from one discrete value to another discrete value. With each state transition, we associate an event. Further, we attribute the state transition to the occurrence of the event. Thus, an event is a specific instantaneous action that causes a state transition and we say that systems that exhibit such behavior are event-driven systems . In continuous-state systems, the system state generally is obtained by solving differential equation representations of the system. In such systems, state changes can occur simply because time advances, even when there is no input to the system. We say that systems that exhibit such behavior are time-driven systems .

Simulation Paradigm Definitions

There are three simulation methodologies, called simulation paradigms , for simulating a model: Monte Carlo simulation paradigm; continuous simulation paradigm; and discrete event simulation paradigm. Selection of a simulation paradigm is based upon the system characteristics associated with the model utilized to represent the simuland. The need for three simulation paradigms is due to the differences in the mathematical properties of the model functional representations. The simulation paradigms are defined in the following.

• Monte Carlo Simulation Paradigm  – The Monte Carlo simulation paradigm refers to the methodology used to simulate static, stochastic system models in which system behavior is represented using probability. The underlying model usually is a random experiment and associated probability space.

• Continuous Simulation Paradigm  – The continuous simulation paradigm refers to the methodology used to simulate dynamic, continuous-state, time-driven system models. The underlying model usually is a set of differential equations that describe simuland behavior. Simulation output often is a time-trajectory of some simuland state variable. The simulation methodology consists of starting from some initial system state and repeatedly solving numerically the differential equations for very small increments of the time variable. This paradigm usually is used for natural systems where it is possible to associate differential equations with system behavior.

• Discrete Event Simulation Paradigm  – The discrete event simulation paradigm refers to the methodology used to simulate dynamic, discrete-state, event-driven system models. The underlying model is usually a representation of a discrete event system [2] such as a queuing model, a state automata model, a Petri net model, or an event graph model. Simulation output often is a sequence of state variable values evaluated at event times. The simulation methodology consists of starting from some initial system state and repeatedly updating the system state at the occurrence of each event. Event management is conducted using an event scheduling strategy in which a future event list is updated at each event time. This paradigm usually is used for service systems where it is possible to associate event descriptions with system behavior.

The process for selecting a simulation paradigm is illustrated in Fig. 6.1. It is clear in this figure that once a model of the simuland is developed, the resulting simulation paradigm that is required to simulate the model is also determined. Often however, there is some flexibility in deciding how to develop the simuland model, thus providing some flexibility in choice of simulation paradigm.

Fig. 6.1

Simulation paradigm selection process

A fourth simulation paradigm, the hybrid simulation paradigm , is sometimes defined; however, this term refers to a simulation methodology that employs concurrently two or more of the simulation paradigms, as defined above, to simulate a single model. For example, the hybrid methodology might be useful when simulating a continuous-state, time-driven model that operates in two different modes. A discrete event system model might be used to change operating modes, while different continuous simulation models might be used to represent system operation in each of the two modes. For example, this situation easily could occur when simulating a model of human physiology.

Selected Modeling Methods

There are numerous methods for developing models and one of the early challenges in any M&S project is the selection of an appropriate modeling method. While each project is unique, there are several guiding principles that apply in all situations. The starting point always is a detailed investigation of the simuland and enumeration of the objectives for the study; that is, identification of the questions about the simuland that the simulation study is to address. The simuland must be modeled so that relevant simuland features are included in the model at a resolution (level of detail) sufficient to address study questions. It is convenient if the model can be developed so that it fits within one of the three simulation paradigms. If that can be done, then there are well-defined procedures for simulating the model and a host of available M&S tools or environments that may be applicable. If the model characteristics do not fit into one of the three simulation paradigms, then a unique simulation methodology must be crafted for that model.

In this section, we introduce two modeling methods, system dynamics models and agent-based models . Both methods frequently are used to describe complex healthcare and medical systems, but each provides a very different perspective of system operation. Both modeling methods address dynamic systems, but can be formulated as either continuous simulation models or discrete event simulation models. In the third section of this chapter, Example Healthcare Applications, both modeling approaches are used to address disease epidemiology. The systems dynamics model is developed as a continuous simulation model while the agent-based model is developed as a discrete event simulation model.

System Dynamics Models

System dynamics models consist of the combination of two components, a stock and flow diagram and a causal loop diagram. A stock is some quantity that is accumulated over time by inflows and depleted by outflows. Stock can only be changed by flows. Thus, stock can be viewed as an integration of flows over time, with inflows adding to the accumulated stock and outflows subtracting from the accumulated stock. Variables representing stock levels usually comprise the state variables for a system dynamics model. A causal loop diagram is a diagram that shows how different system variables and parameters are interrelated. The diagram consists of nodes representing variables or parameters and edges representing relationships between nodes. A positive labelled edge denotes a reinforcing relationship while a negative labelled edge denotes an inhibiting relationship. In system dynamics, the causal loop diagram is used to show how the system state variables and parameters influence the stock inflow rates and outflow rates. The system dynamics model results in the definition of a set of state variable equations describing the dynamical behavior of the modeled system. Ideally, the model state variables and parameters are selected to correspond to specific characteristics of simuland . An example system dynamics model is shown in Fig. 6.3 in the next section.

Numerous applications of system dynamics can be found in healthcare and medical simulation research. In healthcare, the areas of application span disease and substance abuse epidemiology, health care capacity analysis and optimization, and patient flow studies in clinics and emergency care facilities. Examples of disease epidemiology research include heart disease and diabetes studies centering on the impact of prevention and rehabilitation on public health costs [3]. In addition, there have been HIV/AIDS simulation efforts emphasizing virological and behavioral features of the epidemic while portraying the consequences in a simple graphical form [4] as well as the impact of antiretroviral therapy [5]. There also are simulation models for evaluating the possible effects of a screening and vaccination campaign against the human papilloma virus and the impact on cervical cancer [6]. Recent substance abuse epidemiology research centers in particular on cocaine and heroin abuse. For example, a system dynamics model that reproduces a variety of national indicator data on cocaine use and supply over a 15-year period and provides detailed estimates of actual underlying prevalence [7] has been reported. Clinical capacity and flow studies include an optimization study of an Emergency Room [8] in which a system dynamics model is used to investigate the interaction between demand patterns, resource deployment, hospital processes, and bed numbers. One of the findings is that while some delays to patient care are unavoidable, delay reductions often can be achieved by selective augmentation of resources within the unit.

Agent-Based Models

Agent-based models [9] are composed of three components, agents, an environment, and a set of agent relationships or interactions. Agents are self-contained, autonomous objects or actors that represent components of the simuland . An agent has inputs, representing communications from other agents or perceptions from the environment, and produces outputs representing communications to other agents or interactions with the environment. An agent often has a purpose, trying to achieve some goal or to accomplish some task, and the capability to modify behavior over time to improve performance in accomplishing objectives. An environment may be as simple as a grid or lattice structure that provides information on the spatial location of an agent relative to other agents, or may consist of complex dynamic models capable of supplying environmental data that may influence agent behavior. It is the rules of agent interactions, both with other agents and the environment, that are at the heart of any agent-based model. These interactions are usually conducted at the local spatial level with the agents interacting myopically with their immediate neighbors, but can also occur through other environmental projections such as a social network. These interactions might be direct with agents exchanging information, or indirect with an agent deciding to move because it is surrounded by too many neighbors.

It is the combination of many agents interacting simultaneously with each other and with the environment that can lead to emergent behavior within the simulation of agent-based models. Agent-based models are developed at the micro-level through defined agent interactions, but are used to provide insight at the macro-level by observation of the collective behavior of agents. A key property of agent-based models is that even relatively simple rules of agent interaction can result in highly complex collective agent behaviors. Another advantage of agent-based models is their capability to accommodate agent heterogeneity. Agent heterogeneity refers to agents that have different characteristics; they may start with different resources, they may have different tolerances, and they may react differently. The facility for incorporating heterogeneous agents in an agent-based model allows modelers to more closely represent the great diversity that is present in almost all natural systems.

Agent-based modeling primarily is a decision-support modeling methodology. It often is used to develop and test theories and to provide insight into complex system behavior. In the biological sciences, agent-based models have been used to model cell behavior and interaction [10], the working of the human immune system [11], and the spread of disease [12]. Agent-based epidemic and pandemic models can incorporate spatial and social network topologies to model people’s activities and interactions. The focus is on understanding conditions that might lead to an epidemic and identifying mitigation measures. Agent-based modeling is one means to utilize the vast healthcare data pool to analyze the impacts of health-related policy decisions on the general public, especially when it would be impracticable, costly, or potentially unethical to use live experiments to evaluate these policies. Agent-based models and simulations allow researchers to experiment with large simulated autonomous and heterogeneous populations to see what phenomena emerge and to evolve theories about these phenomena.

Example Healthcare Applications

The study of the spread of diseases provides a rich domain for selecting examples to illustrate the significance of choosing a modeling methodology. In this section, we develop epidemiological models using a systems dynamics modeling approach and an agent-based modeling approach. The purpose of these examples is to demonstrate that the selection of a modeling methodology has a direct impact on the level of resolution and the uses that can be made of the information that result from simulating the model.

Heath et al. [13] have proposed three different levels for characterizing models based upon the level of understanding concerning the simuland. The levels are called Generator models , Mediator models, and Predictor models. A Generator is a model developed with limited understanding of the simuland and its use is limited primarily to determine if a given conceptual model/theory is capable of generating observed behavior of the simuland . A Mediator is a model developed with a moderate level of understanding of the simuland and it is used primarily to establish the capability of the model to represent the simuland and to gain insight into the characteristics and behaviors of the simuland. A Predictor is a model developed with full understanding of the simuland and it is used primarily to estimate or predict the behavior of the simuland under various operating conditions and environments. A first step in the development of a conceptual model for a simuland is to select a model methodology. This decision often is based on the (strike) developer’s level of understanding concerning the simuland. It is important to recognize that this decision has a direct impact on how we can use the simulation results.

System Dynamics Approach to Epidemiological Modeling

A basic system dynamics approach to modeling the spread of an infectious disease within a population is known as compartmental modeling . In this approach, the population is partitioned into compartments or subgroups and the model is designed to show how the population of each subgroup changes as the disease progresses. Five different compartmental models are shown in Fig. 6.2. In this figure, each box represents a population compartment and the compartment variable indicates the population of that compartment. The selection of a model is made to best represent the specific disease being studied. For some diseases such as mumps, members of the susceptible population move to the infectious population when they come in contact with another member of the infectious population. Members of the infectious population eventually move to the recovered population and as a result cannot be re-infected. This model is called the SIR model . Other diseases such as strep throat do not grant immunity to those that recover and thus route those recovering back to the susceptible population. This model is called the SIS model . Diseases such as measles provide maternally derived immunity to young infants who do not move to the susceptible population until growing out of the maternal immunity stage. This model is called the MSIR model . Still another model subdivides those in the infectious population into an exposed population where members have been exposed to an infectious person but are not yet contagious. Eventually, members of the exposed population move to the infectious population. This model is called the SEIR model . Other partitions separate the infectious population into a subgroup that is infectious but displays no symptoms and a subgroup that is infectious and displays normal symptoms of the disease. This model is called the SI_CIR model ; Typhoid Mary is a classic example of a member of the infectious carrier population. It is interesting to note that the same compartment definitions can be applied to characterizing the states of an individual modeled as an agent in an agent-based epidemiological model.

Fig. 6.2

Compartmentalized populations for common epidemiological models

We have chosen to use the SIR system dynamics model as an example. The compartmentalized population diagram shown in Fig. 6.2 is used as the stock and flow diagram. Causal relationships, that relate population flow rates between population subgroups to the compartmentalized populations and the flow parameters for infection rate constant b and recovery rate constant k, are added to the stock and flow diagram to complete the systems dynamics model. State variable equations are developed from the model and result in three first-order differential equations that express the time rate of change for the subgroup populations. The complete system dynamics model, including the resulting model differential equations, is shown in Fig. 6.3.

Fig. 6.3

Complete systems dynamics model for SIR example

The SIR model is simulated using the continuous simulation paradigm. We set the population N = S + I + R at 7,900,000 people and it is assumed that N remains constant over the duration of the simulation. It also is assumed that initially ten people are in the infectious population, no people are in the recovered population, and the remaining people are in the susceptible population. The infectious rate constant b is set to 0.50 infectious contacts per day per infected person and the recovery rate constant k is set to 0.33 indicating the fraction of infectious people recovering per day. The simulation is run for a period of 150 days. The simulation results are shown in Fig. 6.4.

Fig. 6.4

Simulation results for SIR system dynamics model

The simulation output for the SIR system dynamics model clearly show how the compartmentalized populations change as a function of time as the infectious disease runs its course. The model facilitates investigating how changes to the initial population distribution, the infection rate constant b, and the recovery rate constant k affect the spread of the disease over time and the portion of the population impacted during the disease lifecycle. However, this model provides no information about how physical interactions between infectious people and susceptible people impact disease spread and the eventual severity of the outbreak. However, such information might be essential if an objective of the study were to identify methods to mitigate the spread of disease.

Agent-Based Approach to Epidemiological Modeling

An agent-based modeling approach presents the opportunity to investigate at greater resolution the causes for the spread of an infectious disease. Our investigations using the SIR system dynamics model showed that disease spread is not due to population subgroup sizes, but rather is due to interactions between infectious individuals and susceptible individuals. Since agent-based models are developed at the individual level, this modeling method facilitates adding much greater detail about how individuals interact.

An agent-based model for the spread of mumps in a small urban environment is presented in [14]. In this model, the agent environment is augmented using geographical information system (GIS) data that identify where individuals live, where they are likely to travel during daily activities, and how they are likely to travel. Individuals are represented as agents. Agent state information includes an activity state, with values representing work/study, leisure, commuting, and a disease state that takes its value from the SEIR states of susceptible, exposed, infectious, and recovered. This state information, when combined with the GIS information, adds considerable detail as to how susceptible and infectious individuals make contact. The flow diagram describing the corresponding agent logic is shown in Fig. 6.5. The flow diagram determines when a susceptible individual comes in contact with an infectious individual and then adjusts the infection rate constant according to the population density at that location.

Fig. 6.5

Flow diagram for infection rules describing disease spread. (From [14])

The model is initialized by distributing the population (1000 individuals) to their home locations in the urban area. In this example, it is assumed that 999 individuals start in the susceptible state and one individual starts in the infectious state. The model is simulated using the discrete event simulation paradigm and the size of the four population subgroups is reported as output. The simulation output is shown in Fig. 6.6. The model allows investigation of how daily behaviors of individuals impact the spread of disease.

Fig. 6.6

Simulation results for SEIR agent-based model . (From [14])

Conclusion

In this chapter, we have presented a brief overview of how computational modeling and simulation can be used to support healthcare research. In particular, we have described two modeling approaches, system dynamics models and agent-based models, commonly used in healthcare research. Examples showing the use of these models in the epidemiology domain are used to demonstrate the importance of selecting an appropriate model; that is, a model having sufficient resolution to address the questions being asked about the simuland.