Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

International interventions require unconventional approaches to modeling and analysis. According to Alberts et al. (2007, p. 5), the characteristics of intervention problems include:

  1. 1.

    The number and diversity of the participants is such that

    1. (a)

      There are multiple interdependent lines of management and control,

    2. (b)

      The objective functions of the participants conflict with one another or their components have significantly different weights, or

    3. (c)

      The participants’ perceptions of the situation differ in important ways; and

  2. 2.

    The effects space spans multiple domains and there is

    1. (a)

      A lack of understanding of networked cause-and-effect relationships, and

    2. (b)

      An inability to predict effects that are likely to arise from alternative plans of action.

In such situations, analysts and planners need to follow a set of principles that are very different from those of situations with unified management structure, a clear objective and a situation understood by all, and an environment that has little adaptation and whose behavior is reliably predicted. First, they must be aware of the numerous arenas and domains involved in complex adaptive systems. Second, because of the lack of predictability in complex systems, planners must take steps to produce agile plans.

Effective computational models hold the promise of enabling planners to explore the deep dynamics of complex situations, and to explore effects across a wider range of candidate policies or plans. By evaluating a wide range of plans across a variety of situations, analysts and planners can achieve more agile and robust strategies that account for the uncertainties in knowledge of the actual situation. Bankes has described this goal of plan robustness as follows:

A level set provides much more information than does a single optimal policy. Combining this idea with that of policy landscapes, the computer can be used to discover policies that are robust across multiple scenarios or alternative models, and to identify and graphically depict sets of policies with satisfactory robustness (Bankes 2002).

Computational models and simulations will specifically allow analysis and planning teams to address the two principles by encouraging the rigorous analysis of complex international actions by explicit modeling in the following ways:

First by structural analysis, the process of decomposing the situation into fundamental components and their interactions, and quantifying the relationships between components,

Second, by the dynamic analysis of the behavior of the system of interconnected systems, using simulation to gain familiarity with the interaction between systems; this includes the analysis of sensitivity of the systems to key factors.

Next, by exploratory analysis of the effects (anticipated and unanticipated) of a range of potential actions by a variety of parties and groups using computational simulations.

This chapter introduces the modeling and simulation technologies available to represent complex situations: the political, military, economic, social, information, and infrastructure (PMESII) states of systems and the effects of diplomatic, information, military and economic (DIME) actions on those systems. First, the methods of explicitly representing complex situations are described, illustrating the methods to translate the tacit knowledge of subject matter experts’ (SMEs) mental models to explicit conceptual models and then to computational models. Second,Footnote 1 We introduce the means by which these models can be applied to represent systems in which physical elements dominate (e.g., infrastructure, etc.) and systems in which nonphysical elements dominate (e.g., the human, social and cultural factors that dominate political, social and economic systems). Then, we explain the uses of these models to estimate the system state, hidden relationships, variables, uncertainties, and dependencies.

The alternative methods of implementing static computational models and dynamic simulations are described, introducing the application and appropriate roles for discrete event, system dynamic, Markov, Bayesian and agent-based modeling implementations. The composition of integrated simulations using these alternative modeling approaches is described. Next, the chapter describes the use of modeling technology to perform exploratory analysis to determine the effects of our actions, to develop effective and acceptable courses of action, and to perform assessments of the models in situ. The chapter concludes with an overview of the issues of model validations, describing approaches to verification, validation, and accreditation in the context of uncertainty and exploratory analysis.

1 Representing Situations

Analysts have long sought efficient methods to describe, with precision, the makeup of complex economic, political, and military situations. Situation Assessment (also called political analysis) is the process used by analysts to identify the key actors (individual leaders, organizations, or aggregate population segments) involved in political competition or conflict over resources, policy, or other aspects of power. The actor’s interests and goals, roles, constraints, abilities, behaviors, and lines of influence to other actors are identified. In addition, the context of the situation (e.g., economic environment, political landscape, cultural-social setting) is described. The process is generally a static enumeration of the situation at a point in time and is generally reported in narrative form with supporting tabular data, where appropriate. (For a detailed list of the elements of a comprehensive situation assessment, see Covey et al. 2005, p. 45). Consider, for example, the typical narrative situation assessments of Iraq in three documents with significant influence on the U.S. policy:

  • Prospects for Iraq’s Stability: A Challenging Road Ahead, National Intelligence Estimate, ODNI, January 2007. The unclassified judgments in this national intelligence estimate include three pages of narrative assessment, followed by a half-page judgment on three “security paths” or adverse trajectories that could occur if violence does not subside (ODNI 2007).

  • James A. Baker, III, and Lee H. Hamilton, The Iraq Study Group Report, NY: Vintage, 2006. This report includes a 30-page narrative assessment, followed by a 3-page projection of the consequences of a continued decline in security in Iraq. This assessment preceded a 60-page analysis of alternative courses of action and recommendations (Baker and Hamilton 2006).

  • Stabilizing Iraq: An Assessment of the Security Situation, Statement for the Record by David M. Walker Comptroller General of the United States, GAO-06-1094T, Sept. 11, 2006. This document, supporting congressional testimony, includes an 18-page narrative assessment, supported by two graphs of violent incident trends and two tables of data on Iraqi security force readiness (Walker et al. 2006).

In each case, the analysis enumerates the major factors and the interrelations between the factors and provides a narration of possible scenarios (i.e., the dynamics of alternative outcomes). This process of decomposition (breaking apart, or factoring) identifies the component parts (or subsystems) and their interconnections to allow the situation to be more easily understood, analyzed, and described. It also allows individual factors (e.g., politics, economics) to be described in greater detail.

Modeling technology is now allowing us to go beyond these static enumerations and narrative descriptions of potential scenarios and effects, even as analysts (who seek to understand a situation) and planners (who develop approaches to change the situation to achieve an objective) are seeking more breadth of enumeration and depth in dynamic analysis. A planner identified the need for more effective means of analysis and planning:

Analytical tools have improved dramatically. Unfortunately, questions over effects-based [approaches to] operations persist: the adequacy of intelligence, the lack of cultural sensitivity, the risk of studying inputs rather than outputs, and the need for models to account for cognitive, cultural, political, and social factors. These are serious questions, and their solutions are not obvious (Meilinger 2004, p. 122).

Solutions would include a rigorous process to decompose and represent the behavior of a real-world system, S (e.g., a regional political-military competition between states, a single nation-state, a provincial insurgency, or the stabilization of a major urban area), which comprise interdependent physical and nonphysical (e.g., social, economic) contributing elements with behaviors in component models, m, and interactions, i, in a composite model, M, such that:

(Completeness) The decomposed set of m and i, once composed into M, can be shown to achieve a measurable level of coverage, C, of the elements and behaviors of S to represent a specified level of causal granularity, G.

(Behavioral Specificity) The component models and interactions between models in M can be specified to achieve an aggregate level of G, and the dynamic behavior of M can achieve a specified degree of the behavior of S.

(Descriptive causality) The level of G can be related to specific causes and effects ­achievable in M and observable in S.

Note that this challenge presumes decomposability of S to some degree; if all elements at the finest granularity are independent in some significant degree, then decomposition at a higher level is not possible (Table 1).

Table 1 Representative PMESII elements and aspects
Full size table

A representative process of decomposing a situation into PMESII elements and then interacting computational models proceeds by decomposing the situation into key elements (or systems) and their interactions and then composing models that represent the situation by these elements and their interactions. The process proceeds in the following steps (Fig. 1):

Fig. 1
figure 1

Situation decomposition and model composition

Full size image
  1. 1.

    Describe the situation informally by discussion with SMEs who can enumerate the key elements (actors and systems), their relationships and interactions, the critical factors of influence, and the behaviors of these elements. These discussions are often in narrative form (stories), and the quantification process requires careful translation of the SMEs’ qualitative narratives into conceptual models. In this step, it is also critical to recognize if there are multiple concepts (hypotheses) of how a situation operates. For example, one group of SMEs may believe that a nation is driven by its underground economy and external influence, but another group may believe that it is driven by the official economy and internal cultural factors. In these cases, it may be necessary to maintain both models: two hypotheses that may be used to evaluate plans, in order to develop robust plans that can address either of the two views of how the country operates.

  2. 2.

    Identify the Elements, Relationships, and Systems. From the initial discussions, guided by the typical PMESII factors, develop conceptual representations (generally, tabular lists of elements and graphical depictions of relationships) of the major elements; review these with SMEs and refine until the SMEs agree that these conceptual models represent the structure of the situation. Also represent the major dynamics of the situation (e.g., “legitimate economy will go up as the illegitimate economy goes down and corruption is reduced”) and confirm these major behavioral factors with SMEs.

  3. 3.

    Develop Component Models of System Structure That Will Produce Expected Behaviors. The component models of PMESII subsystems (e.g., the legitimate and illegitimate, or underground economy subsystem models) are created and tested to produce the behavior expected by the SMEs. The models are evaluated for a range of behaviors, using historical data when available, and by the SMEs to compare model behavior to SME-expected behaviors.

  4. 4.

    Compose the Component Models into a Metamodel. Finally, the models are ­composed (integrated) into a unified model of models; the interconnections between models (e.g., the agricultural impact on the legitimate economy and the drug-crop interconnections with the illegitimate economy, and their interactions) make up, in fact, a model in its own right. This metamodel of interconnections will produce large-scale system behaviors that are not inherent in the independent models, producing effects that emerge from the interaction of the models. At this stage, the measures of systems performance (metrics) that characterize the situation must be defined and checked to verify that the model can be compared to the real situation; this will also aid in the identification of means in the real world that can be used to compare model results to real-world situation dynamics.

2 Conceptual Modeling

In the preceding section, we used the general term model to refer to any abstract representation of a system, but we now distinguish among:

  • Mental models of systems or phenomena that are understood (or believed to be understood to some degree) by the SMEs.

  • Conceptual model representations of mental models that may be presented in a variety of narrative or graphical means to explicitly represent elements, relationships, and causal functions of a system or phenomena.

  • Computational models that implement the structure and causal behavior of a conceptual model in a computational form that allows the dynamic behavior of the modeled system to be simulated.

The process for representing a given situation, using the PMESII categories, proceeds from the tacit mental models of experts to computational models that allow analysts to explore the dynamics of interacting PMESII systems (Fig. 2). The process of abstraction – representing the real (concrete) world in qualitative structures and quantitative functional relationships – requires the capture of SMEs’ tacit knowledge of the particular PMESII systems of an area in explicit conceptual models. These models are first captured in narrative form, and in lists of enumerated actors, systems, and dependent interrelationships. From these lists, graphical structures are created at a common level of granularity. An appropriate computational modeling paradigm (approach to modeling causality, e.g., Bayesian model of influence, Petri net model of a sequential process, system-dynamics model of process flows, agent-based model of social behavior) is selected. The computational model is constructed and operated over a range of environments and perturbations to evaluate and refine its behavior relative to known behaviors in the real world.

Fig. 2
figure 2

The relationships between mental, conceptual, and computational models

Full size image

The upward process in the figure illustrates how the results of computational experiments flow upward from computational model results to conceptual displays of the dynamics of conceptual model variables, in a form understandable to the modeler and analyst. The results of experimental simulation refine the analysts’ and SMEs’ mental models as results are questioned and are used to refine the models until confidence is built in the results and models become useful for exploring large-scale dynamics. The quantitative results can again be translated into narrative “stories” that describe potential outcomes of candidate actions.

It is important to distinguish between empirical modeling and the kinds of causal models that we apply in this chapter (Fig. 3). Empirical modeling refers to those methods that represent a system or phenomenon based on the data produced from prior experience or experimentation. Generally, this involves quantitative methods of regression that seek to establish the functional relation between selected values of x and observed values of y (from which the most probable value of y can be predicted for any value of x). In the figure, the use of direct regression (path 1) can produce a functional model that can extrapolate future values as a function of the past. Causal modeling (paths 2 and 3 in the figure) seeks to induce the underlying causality (functional processes) of phenomena or systems and represent them explicitly. Such a causal model can then create representations of future behavior deductively from input variables, and the internal system behavior can be explicitly observed and compared to the real world.

Fig. 3
figure 3

Distinguishing empirical and causal modeling approaches

Full size image

Causal modeling often proceeds from the narrative model of an SME to a corresponding graphical causal model (example, Fig. 4) that represents the major elements (actors, systems, processes, etc.) and the structure (relationships between elements) of the model. The process for causal modeling often proceeds:

Fig. 4
figure 4

Translating a narrative conceptual model to a corresponding graphical conceptual model

Full size image
  1. 1.

    SME is interviewed to describe the system and its major elements, the factors that influence its behavior, and the key relationships between elements. A narrative description is developed, with a list of elements and relationships for review by the SME.

  2. 2.

    The underlying empirical data is sought to develop the empirical basis for the current situation; the accepted theoretical basis is also sought to develop the relevant causal model.

  3. 3.

    A corresponding graphical (functional) model is developed and again described to the SME to refine the modeler’s understanding of the system. This may be a much different perspective of the same system the SME knows well, and the discussion may reveal more insight as the SME is asked to detail more explicitly the causal behavior, thus refining the conceptual model. (This process is the beginning of internal model validation: building confidence in the underlying theory on which the model is based.)

  4. 4.

    A computational model of the system is developed and the behavior of the model over a range of conditions is recorded and presented to the SME to perform a comparison to known real-world behavior. Discrepancies between the model output and empirical data must be examined, explained, and the model refined until the model behavior compares to the real world sufficiently for its intended use. (This process is the beginning of external model validation: building confidence by comparison of the model behavior to the SME’s empirical understanding of real-world behavior and, if appropriate, historical cases.)

Of course, the preceding method is nothing more than an instance of the general procedure of the scientific method, which is based on hypothesis (model) building, prediction of behavior, and testing against empirical data.

Consider, for example, three approaches to decomposition of the primary systems that represent the competitive structure of an insurgency and counterinsurgency (COIN) and their representation in high-level (of abstraction) conceptual models (Fig. 5). The first decomposition is Manwaring and Fishel’s SWORD model that decomposes the competition into principal actors and their interrelationships using seven dimensions: (1) military actions of the intervening power, (2) support actions of the intervening power, (3) host-government legitimacy, (4) degree of outside support to insurgents, (5) actions against subversion, (6) host-country military actions, and (7) unity of effort (Manwaring and Fishel 1992, pp. 272–305).

Fig. 5
figure 5

Two representative conceptual models of counterinsurgency

Full size image

While the SWORD model identifies key static indicators, further insight into the dynamic modeling requirements can be found in conceptual insurgency-COIN models developed by McCormick and Lynn (McCormick 1999; Lynn 2005) that describe the elements (entities) and relationships between insurgent and COIN forces (Fig. 5b). The essential elements (entities or actors) of both models include:

  • Insurgent Force(s): The leadership, combatants (guerillas), financers, and supporting population that carries on the insurgent political message and a coordinated campaign of violence to undermine the legitimate government and demonstrate its inability to provide security and services.

  • Government and COIN Force (s): The leadership, combatants (military), and supporting population that endorses the current government, its political message and legitimacy. The government carries out a COIN campaign of information and action to support its legitimacy and demonstrate its ability to provide security and services to the population.

  • Civil Population includes elements that support the legitimate government, those that support the insurgents, and the population in the middle for which both sides compete to prove legitimacy and gain support.

  • External supporting powers include those external parties (states, organizations, etc.) that supply ideological, financial, material, or human resources to either side of the conflict.

Both models also represent the basic relationships between these entities by simple arrows that describe the interactions between key actors. The graphical representations of the conceptual models distinguish the actors (leaders or elites, organizations or institutions, and population groups) and the relations between the actors. In both models, government and insurgents compete for population support, and the competition is conducted across the many relationships that exist between the parties (political, military, economic, etc.). The U.S. Army’s Field Manual for COIN acknowledges the value of narrative insurgency models of history and a conceptual modelmaking process for understanding the COIN environment (US Army 2006, p. 1–4, para. 1–76 and p. 4–3, para. 4–9).

3 Computational Modeling

Computational models include a wide range of models that compute output functions as a result of inputs. Computational models include the computation of complex yet static functions (such as a computer spreadsheet) or dynamic simulations that implement models as they operate over time. Simulation tools provide a means for analysts and planners to be immersed in the modeled structure, dynamic behavior and responses (effects) to courses of action; simulation provides a tool for experimentation and exploration of behavior.

The modeler may choose from a variety of computational approaches to implement the component models and to compose them into an integrated model. The primary computational modeling approaches to simulate processes over time (Table 2) are distinguished by three characteristics:

Table 2 Representative modeling and simulation tools (Waltz 2006)
Full size table

  • The method used to move the model through time:Time-continuous functions may be represented in time-discrete steps (time-sampled), and the simulation proceeds to compute all functions and interactions in a time-discrete (incremental step-by-step) fashion; in this case, the unit of simulation progress is a time clock, and all models apply a uniform time constant to represent processes that occur within the time step. Alternatively, the unit of progress in the simulation may be chosen to be discrete events; event-based simulations increment from event to event by event triggers that represent the causal propagation from any given event to any other event-producing processes.

  • The approach to deal with process functions and functional interrelationships: The number of functions (N), functional complexity (C), their interrelationships (R), and the relative autonomy of functions (A) that characterize a model distinguish the models that are relatively compact system-process models (e.g., system-dynamics models in which N, C, R, A attributes are low, high, low, low, respectively) and highly interactive models (e.g., agent-based models in which N, C, R, A attributes are high, low, high, high, respectively).

  • The approach to deal with uncertainty:The uncertainty in inputs to the model and uncertainty in the internal model functions themselves influence the uncertainty in simulation outputs. The models may directly represent and propagate uncertainty throughout the model (e.g., Bayesian networks use probabilities to propagate uncertainty) or a deterministic model may be used to represent uncertainty by varying input or internal variables in a controlled manner (e.g., by being sampled from a distribution) across many simulations to assess the effects of uncertainty. This approach is called Monte Carlo simulation.

For each model category, there exist commercially available model-building tools that allow the modeler to develop, test, and validate models with available empirical data (see Table 2 for representative commercial tools). The characteristics of the major computational modeling approaches are summarized in the following paragraphs.

General Causal Models:A fundamental interest in modeling is the representation of the causal relationships between entities or events. (One event, the cause A, must be prior to or simultaneous with another event, the effect B.) Models of causality are represented by directed acyclic graphs that represent events as nodes and edges (or links) as the causal relationship. From this simple representation, a number of sophisticated model implementations can be created:

Influence Models:Directed graphs that represent functional relationships (influences) between variables are called influence diagrams and are computed as influence models. Such models are often used in decision modeling, and the graph proceeds from decision nodes (alternatives that can be chosen by a decision-maker) and independent variable nodes (deterministic or probabilistic variables) to the dependent objective node: the function influence by decisions and variables that is to be optimized. In simulations, these models may be used to represent the effects of the decisions of rational human decision-makers seeking to optimize an objective.

Bayesian Network Models:These models can represent probabilistic causation, allowing an effect to be probabilistically related to a cause. The models permit effects to be represented by conditional probabilities; for example, P(B|A) represents the probability of the occurrence of the effect B, under the condition that A occurred. Bayesian networks allow the calculation of the propagation of probabilities across complex directed acyclic graphs to compute posterior probabilities, as a function of prior and computed conditional probabilities (Pearl 2000).

General Discrete Event Models:In these causal models, a system’s behavior is represented as a sequence of events in which the triggering of each event can be determined by external conditions or conditioned by the state of other events. These models are implemented as simulation tools that represent systems in which the state evolves at discrete points in time (events) rather than continuously as in time-discrete models. Discrete-event models readily represent processes with transaction, flows, delays, and queues; they are well suited to traffic, production, inventories, and movement of commodities (Banks et al. 2004).

Markov Models:These models represent the dynamic of systems by their states, the state-transition probabilities that move the system from state to state, and the available information to observe the state. A Markov model represents a system with the state directly observable, and a Hidden Markov Model represents a system in which the state is not directly observable (although it may be partially observable or inferred from variables related to the state). These models are used to model systems that move from discrete state to state in their operation.

Petri Network Models:These network models represent interactive and distributed processes, modeling the communication, synchronization of messages and processes, and sharing of resources across distributed processes. Models are represented in a directed graphical notation that represents communication flow and the flow of process activities, represented by tokens being passed across the network. (When the tokens take on value, the nets are referred to as Colored Petri nets.) These models are applied to representing political policy processes that follow legislative sequences, logistic processes, communication distributed computing (network) processes, and commodity delivery processes.

Discrete Time Models:Many models are implemented to operate in discrete time steps, in which the simulation unit is a fixed time interval (as opposed to discrete event simulations in which the simulation units are events that may have varying time intervals). Processes represented in differential equations are well suited to direct implementation as time-discrete models that are updated at the Δt interval. The general causal models previously described are often implemented as discrete-time simulations, as well as the models that follow.

System Dynamics Models:The fundamental principle in representing systems in this model form is the dynamic flow of critical “stocks” in the system modeled; stocks are accumulated or depleted over time (the “flow” of capital or stock). Stocks can refer to material entities (e.g., crops harvested, children born, steel produced) or more immaterial entities (shares of securities owned, financial capital invested, human or intellectual capital, etc.). In this modeling paradigm, the modeler must identify the key stocks that represent the fundamental flow dynamics of the system. For an insurgency organization model, for example, the stocks may be financial capital, insurgent fighters, and weapons; the flows are a function of donations-expenditures for weapons, recruitment and attrition of insurgent fighters, and weapons purchased-weapons consumed or expended, respectively. Once the fundamental stock and flows are defined, the functions that influence the flows are modeled to “throttle” the accumulation and depletion of stocks. These functional relationships allow the modeler to represent critical time delays, queues, and feedback loops that provide positive reinforcement (growth) or negative reinforcement (balancing) behaviors. The completed model provides a simulation of the time-dynamic behavior of the system, the changing level of the critical stocks that describe the system, and the effects of initial conditions and the time delay and feedback functions. The models can readily simulate nonlinear systems and can simulate general equilibrium behavior that is exhibited by economic, production and social systems, as well as the conditions that disturb such stability. Numerous modeling tools allow the model to be created graphically and simulated rapidly, using a standard system dynamics graphical formalism. The graphical symbols are compiled onto ordinary differential equations that represent the flows and conditioning parameters that represent the time delays and feedback loops that couple the differential equations. For a comprehensive overview of system-dynamics models, see Sterman (2000).

Agent-Based Models:These models represent the interaction of a network of autonomous actors, interacting with an awareness of their environment and individually operating by an internal behavior (goal-directed, able to cooperate or compete with other actors). The actors in real life may represent leaders, organizations or the aggregate behavior of population groups, and they are represented by software agents that perceive their environment (e.g., sociopolitical, economic, security, or other aspects), reason about the situation compared to their interests and goals, perform decision-making, and then act in the environment to respond to the situation. Agent-based simulations are described as generative because they autonomously generate behavior as a result of the interaction between agents, generating equilibrium as well as the emergence of higher order (complex) behaviors, not predictable in the behaviors of the individual actors. These simulations uniquely allow the modeler to represent individual and group decision-making to simulate the effects of interactions between large numbers of actors in a dynamic environment. In particular, models that employ agents with relatively modest rules can produce relatively complex behaviors, due to the high level of interactions within the network of agents. The models are most often applied to political and social modeling (e.g., political power struggles over policy positions and social interactions between groups or groups and elites), as well as modeling economies, logistics, and transportation behaviors, and the spread of disease. Because they explicitly represent the decision-making of individuals or groups, they are well suited for the study of organizations. Tools for creating agent-based models include NetLogo (Northwestern University), Swarm (Santa Fe Institute), and Power Structure Toolkit (Soar Technology). For an overview of agent-based models, see Epstein and Axtell (1996) and Axelrod (1997).

Hybrid simulations:Because each modeling approach has a particular strength, it may be appropriate to implement a simulation that integrates (or composes) different types of models to apply the advantages of each. This is often the case when modeling situations in which the interactions of political and social systems, economic systems, and physical systems (computer systems, production, infrastructure, transportation) must be represented. The next paragraphs describe the approaches to model composition to develop hybrid simulations.

4 Model Composition

Once a major system (e.g., an unstable or failing state) has been decomposed into component subsystem models (e.g., PMESII), the analysis of effects across multiple models requires that the individual models be composed into an overall system for analysis. This issue of integrating diverse component models into a composite model has long been a challenge to the modeling community (Davis and Anderson 2004). Consider the two alternative approaches to composition of multiple models.

  • An analytic composition process runs models independently in time but considers the interaction effects by running model excursions to describe the effects of interdependencies. The results are composed by an external analysis of the independent simulation dynamics and basing one model’s inputs on the results of others, but the models do not directly interact.

  • A computational composition process integrates multiple component computer models of individual PMESII systems into a single metamodel (or metasimulation) that describes a larger situation than any one component and synchronizes their interacting operations at the same run-time. When the components are of varying resolutions (or causal granularities), the metamodel is a multiresolution model (MRM). The composed MRM structure may represent a hierarchy of fine-granularity submodels that contribute upward to a lower-granularity model that integrates the results, or lower-granularity models may provide contextual information downward for finer-granularity models.

A computational composition of models has been performed in the system called COMPOEX (COnflict Modeling, Planning, and Outcomes EXperimentation). It is an example of a large-scale simulation framework that composes a diverse set of modeling paradigms into a single run-time metamodel (Fig. 6). The COMPOEX tool architecture (Fig. 6) includes:

Fig. 6
figure 6

Example of a COMPOEX composed model architecture (Waltz 2008)

Full size image

  • A planning tool that organizes and schedules the injection of actions to models along the simulation time sequence, and

  • An option exploration tool that hosts the integrated model and runs simulations of the synchronized sets of composed models.

All models are plugged onto a “backplane” that represents the state vector of PMESII state variables. The models are stepped in time-discrete manner, generally in 1-week increments, simulating behavior over a 2–3 year period of time. Characterizing the integrated simulation as a finite state machine, the state vector is the memory that stores current state; the sequence of states for any given variable over 156 weeks of a 3-year simulation represents the behavior of the variable. A typical COMPOEX model may include well over 10,000 such state variables. The visualization service allows users to customize views of any of the variables and their relationships; it also detects and displays discrete effects that should be brought to the attention of the planner. It furthermore allows the user to trace causality within the simulation, allowing the user to trace the (upstream) variables on which an effect is dependent and the (downstream) variables that are dependent on the effect variable (Waltz 2008).

The model of power actors and relationships is at the core of the COMPOEX simulation, providing the major abstract dynamic within a virtual world of economic, material services, media and sources of information exchange, physical violence, and infrastructure. Power struggle behavior is included across the many composed models within the simulation environment. The COMPOEX approach to abstraction is based on two major partitions of the model:

  • Power Influence Network:Competing actors for power are represented in agent-based models in which autonomous agents compete for power, represented as the abstract capital commodity in four dimensions (political, social, economic, and armed military). This network represents all human decision-making, influence, and action. The operation of the agent-based actor simulation is described in more detail in Taylor et al. (2006).

  • Virtual World:The context within which the actors compete (or cooperate) for power is represented by a set of interconnected process models, implemented by a variety of modeling paradigms (e.g., system dynamics, discrete time models, Bayesian networks). These models may represent aggregate human behavior (e.g., aggregate economics, production, large-scale population behavior), but do not represent the core competition for political power.

The structure of the composed power network and virtual world models (Fig. 7) illustrates the interaction between the actor net and the virtual world. The agent-based actors perform goal-directed behavior to compete in the power struggle; each actor behaves to achieve political, social, economic, and armed power (capital) objectives relative to all other actors in the simulation (Waltz 2008).

Fig. 7
figure 7

Integrating the power structure competition and the virtual physical world

Full size image

A composed metamodel may be organized into multiple levels of causal granularity, such that lower (finer) levels of granularity produce results that influence higher (coarser) levels; the higher levels may also set contextual variables for the lower level models. Consider three typical levels of granularity in such simulations:

  • National or regional level:A top-level country-level model sets the context for the lower-level models, representing country-level political policy, national power struggles and economic base.

  • Province:The overall behavior of individual provinces – political, social, economy – representing the dynamics of the political power struggle, behavior of the social populations, and relations between provinces.

  • City:Major urban areas may be modeled individually (local political struggles, economic powers, civil health services, infrastructure, etc.) and are aggregated upward to the province city level.

All models interact by exchanging variables at common time increments, across a common state vector of variables that represents the PMESII state of the system at any time increment; the MRM operates as a time-discrete state machine allowing models of various modeling paradigms (e.g., agent-based, Bayesian, Petri net, system dynamics) to plug and play on the PMESII state vector.

Large-scale computational PMESII models are excellent candidates for high-performance computing implementation. Such highly-interactive models must be interpreted in the context of the uncertainty inherent in model parameters and system interactions, requiring behavioral uncertainty to be observed over large ensembles of runs (using Monte Carlo methods) that may be distributed across computing nodes on cluster before (1) analyzing the statistics of results to understand aggregate behavioral dynamics and (2) mining all results to discover emergent properties of the complex interactions interaction.

5 Exploring with Models

We are careful to distinguish two desired capabilities when we apply computational models of intervention situations to conduct analyses of internal dynamics and the effects of a potential plan of action:

  • Prediction:the ability to foresee a specific, individual future event or scenario; generally, prediction refers to a high degree of accuracy of outcomes for specified model fidelity, resolution, and granularity.

  • Anticipation:the ability to foresee a “landscape” of feasible futures, an “envelope” or “range” of many point predictions. This allows us to explore the range of dynamic behaviors, feasible events, and consequences, providing awareness of emergent situations that would surprise us if we had not simulated them.

Indeed, the results of computational experiments are, in fact, exploratory: lacking the specificity expected from physics-based model prediction of well-established physical processes (e.g., prediction of the trajectory of the bullet). The PMESII modeling processes track currently available information to drive causal simulations to create an envelope or landscape of many feasible outcomes. The simulations ­create an envelope of other parties’ decisions, actions, and effects, then estimate our courses of action and effects. Simulation tools allow analysts to explore predictive envelopes, not point predictions. The models of decision-making and physical activities are refined over time, and the accuracies of the predictive envelopes can be tracked over time to estimate their predictive performance.

The challenge of intervention modeling, then, is to create explicit models that, by exploration, will reveal assumptions, explicitly show interactions, and simulate complex dynamics of PMESII systems to help the user understand the critical instabilities, potential domains of and emergent chaotic behaviors not expected by the tacit intuition of SMEs. Anticipating PMESII system of systems behavior requires a description of the behavior of humans with free will, organized in social networks, with varying beliefs, desires, motivations, perceptions, and goals. A realizable PMESII prediction methodology confronts the challenge of explaining social systems that exhibit unknowable causality. Jervis has pointed out how the high degree of interaction between policymaking actors in such situations confounds analysis and causal prediction: (1) results of the system cannot be predicted from separate actions of individuals, (2) strategies of any actor depend upon the strategies of others, and (3) the behaviors of interacting actors even change the environment in which they interact (Jervis 1997a, b).

Complexity is the emergent property of social system behavior, caused primarily by the interactions of its independent actors, rather than on the properties of the actors. This behavior cannot be predicted by models of the properties of the actor or by a linear combination of them. Some such linear systems exhibit responses that may have a predictable range of responses (to some degree) or not; other deterministic nonlinear systems exhibit such sensitivity to initial conditions that they exhibit behavior described as chaotic (Gleick 1988). The approach to studying such problems is not analytic (decomposition to reduce to a closed form solution); rather, it requires a synthetic approach, whereby representative models synthesize (simulate) behavior that may be compared to the observed world and refined to understand behavior in a more holistic manner.

6 Building Confidence in Models

PMESII models are not excluded from the necessity to provide a means for users to develop confidence in their validity in order to provide analytic value. Without user confidence in their faithful representation of reality and credible simulation of system behaviors, such models will fail to gain user acceptance, and users will return to the “tried and true” methods of situation analysis: oral discussion and the traditional enumeration of factors and narrative description of plausible scenarios.

Model developers apply a process of confidence-building in the credibility of a model by evaluating the model against two criteria to determine how faithfully it represents reality, for the intended application:

  • Internal criteria:Behavior of the model is consistent with a theory or understanding of phenomena or causality (i.e., the model is internally consistent with a coherent explanation of a system and its phenomena; this theory should be an accepted general theory of structure and behavior).

  • External criteria:Behavior of the model output is consistent with observed real-world behavior (i.e., the model is consistent with at least one particular instance of such a system observed in the real world; it is preferable, of course, if the model can be shown to be consistent over a wide range of conditions, if such data are available).

The formal validation process determines the degree to which a model or simulation is an accurate representation of the real world from the perspective of the intended uses of the model or simulation (emphasis added). This definition focuses on the external criteria (DoD 1994). (The process is distinguished from verification, the process that precedes validation to evaluate the correctness of a model with respect to a certain formal specification of a theory, using the formal methods of testing, inspection, and reviewing.)

When considering validation of PMESII models, it is important to distinguish between those models that are used as a substitute for thinking and those that serve the purpose of stimulating deep thinking (Table 3). A fire-control computer, for example, uses physical kinematic models to compute ballistic trajectories in support of an artillery officer by eliminating the need for thinking about trigonometry. In contrast, the models described in this chapter are for analysts and planners and serve the purpose of aiding them to think deeply and broadly about the structure and dynamics of a situation and the effects of alternate actions. In this case, validation is not performed once and trusted thereafter. The very phenomena of social situations remain in flux, and the validation process must often be performed in situ, on a day-to-day basis. In the earlier case, gravity, ordnance mass, and the influence of other physical factors remain constant; in the case of models of human, social and cultural systems, the entities have free will and the modeler cannot count on a constant human behavior.

Table 3 The roles of validation in modeling
Full size table

A recent RAND study described the basis of validation in such models, in which uncertainty in the model (e.g., application of a particular theory of social behavior and response to media appeals) and in the source data inputs (e.g., uncertainty in demographic data on tribal affiliations) is deep:

[Our conclusions] apply when the models or their data are more afflicted with uncertainty. For example, no one has a “correct” model of war with all its notorious complications, and, even if such a model existed, it would have large numbers of uncertain inputs. … In such cases, we believe that model validation should be construed quite differently than might be suggested by the usual definition of validity. A validation process might reasonably conclude by assessing the model and its associated databases as “valid for exploratory analysis” or “valid, subject to the principal assumptions underlying the model, for exploratory analysis” (Bigelow and Davis 2003).

Similarly, the National Research Council (NRC) has recognized that the techniques used to validate models in the physical sciences are not appropriate for modeling the behavior of individual, organizational, and societal (IOS) systems:

Verification, validation, and accreditation: These important functions often are made more difficult by expectations that verification, validation, and accreditation (V&V) – as it has been defined for the validation of models of physical systems – can be usefully applied to IOS models. … Current V&V concepts and practices were developed for the physical sciences, and we argue that different approaches are needed for IOS (individuals, organizations, and societies) models (Zacharias et al. 2008).

The RAND report by Bigelow and Davis (2003) on validation of multiresolution models concluded that comprehensibility, explainability, and uncertainty representation are the critical elements for such models:

The authors believe that when working within this troubled but common domain, it is particularly important for two criteria to be met in assessing a model (and its associated data):

  • The model should be comprehensible and explainable, often in a way conducive to explaining its workings with a credible and suitable “story.”

  • The model and its data should deal effectively with uncertainty, possibly massive uncertainty.

Referring to the use of societal models (note that in this book the terms PMESII model and societal model are used interchangeably) to increase our understanding of complex system behavior, a pioneer of social modeling wisely noted: “The moral of the story is that models that aim to explore fundamental processes should be judged by their fruitfulness, not by their accuracy. For this purpose, realistic representation of many details is unnecessary and even counterproductive. … the intention is to explore fundamental social processes … the interactions of adaptive agents typically lead to nonlinear effects that are not amenable to the deductive tools of formal mathematics.” (Axelrod 1997, p. 6).

7 Summary

Emerging analytic and planning tools allow analysts and planners to capture models of interventions and help anticipate their effects. We distinguish between mental models of systems or phenomena; conceptual model representations of elements, relationships, and causal functions; and computational models that implement a conceptual model and simulate the time-dynamic behavior of the modeled system. Empirical modeling represents relations between variables of a system or phenomenon based on the data from experience or experimentation, e.g., via regression methods. Causal modeling explicitly represents underlying causality (functional processes) of phenomena and derives future behavior deductively from input variables. Computational simulations of interventions are developed by decomposing the political, military, economic, social, information and infrastructure (PMESII) elements of a situation, and then representing them in component models. Modeling techniques for representing human-social systems include agent-based models, Bayesian network models, time-discrete and event-discrete models, system dynamics models, and Markov and Petri models. A model composition framework is required to integrate diverse models. For example, the DARPA COMPOEX (COnflict Modeling, Planning, and Outcome EXperimentation) program developed a large-scale simulation framework and an associated PMESII model component library, and demonstrated an ability to compose a diverse set of modeling paradigms into a single run-time metamodel. Large PMESII models face the validation challenge of demonstrating that they represent the real world well enough to support their intended uses. The uses may include prediction, i.e., the ability to foresee a specific, individual future event or scenario; or anticipation, i.e., the ability to foresee a “landscape” of feasible futures. In particular, such tools can offer awareness of emergent situations that would surprise us if we had not simulated them. The tools can aid the validation process by permitting analysts to compare modeled behavior to situation data and to refine both their models and their understanding of the systems and phenomena they represent.