1 Introduction

Modeling the behavior of wildland fires is challenging due to the complex set of coupled processes that drive the characteristics of a spreading wildfire and the large range of spatial and temporal scales over which these processes occur [1, 2]. Predictions of crown fire behavior, i.e., fires that involve the combustion of canopy biomass, are particularly difficult [3, 4] but of significant importance given that these fires have greater rates of spread, flame lengths, and increased fire brand generation and transport compared to surface fires [5, 6]. The increased fire behavior associated with crown fires can decrease the effectiveness of fire suppression actions [7] and can have major ecological impacts.

A number of mathematical approaches have been used in the development of fire behavior models, ranging from empirical to theoretically-based, to account for the various interacting processes that drive fire behavior and to overcome computational and time limitations associated with specific applications. Empirical models are those that pose mathematical functions (usually algebraic) to describe the relationship between macro-scale fire behavior and environmental conditions based on observed data. In contrast, theoretical or process-based models use mathematical expressions (often partial differential equations) to describe individual processes that drive fire behavior. Sullivan [8–10] provides description and overview of recent wildland fire models.

The most commonly used crown fire behavior models are largely empirical and focus on point predictions of crown fire rate of spread [e.g., 7, 11] given the importance of this variable in wildland fire management. As noted by Chandler et al. [12], such models are primarily used for fire suppression planning purposes, but they have also been widely applied to address a variety of research questions. However, as suggested by U.S. National Science and Technology Council’s Subcommittee on Disaster Reduction [13], there is a need to advance wildland fire models so that they capture the physical processes impacting natural resources, the environment, and physical infrastructure. This need has led several researchers to developing more detailed physics-based models [2, 14–16]. In contrast to empirically derived models, detailed physics-based modeling approaches attempt to simulate the behavior of a system with a set of functional components and their interactions with each other and with the system environment through mathematical representations of the physical and mechanistic processes driving fire behavior over both space and time [17, 18]. In this paper, we use the term detailed physics-based model to denote a computational fluid dynamics (CFD) based numerical model that solves a set of three-dimensional time dependent governing equations which describe the evolution of and interaction between the recognized processes driving fire behavior for the application of interest (i.e., crown fire simulation here). These models typically attempt to represent all of the critical physical processes to some degree even though they do have to employ some approximation to account for processes that occur at length scales below those explicitly resolved.

Morvan [19] provides an overview and some examples of detailed physics-based approaches for simulating wildfires. Of the four approaches discussed by Morvan [19], we consider two here: FIRETEC [15] and WFDS [2, 20]. These two models were chosen in part because they can simulate fire behavior in three-dimensional domains large enough to contain a crown fire (i.e., a volume 100 s of meters on a side) and in part because they have been utilized to predict crown fire behavior in a number of published studies [21–26]. The two other detailed physics-based approaches considered by Morvan [19] have only been applied, to our knowledge, to fires in smaller domains or are limited to two dimensions. Advantages that three-dimensional detailed physics-based models have compared to simpler empirical models are that they can (1) represent the three-dimensional nature of the entire fuels complex [23, 25, 27]; (2) account for potential nonlinear dynamics arising from interactions among the fire, atmosphere and fuels complex [27]; (3) provide predictions of additional fire behavior quantities (e.g. heat fluxes, temperatures, gas velocities) beyond the rate of spread without linkages to additional models; and (4) be used to predict fire behavior outside the environmental conditions for which we have experiments [25]. Although detailed physics-based models provide the user with a great deal of flexibility to investigate a variety of wildland fire behavior questions, there is a need to assess model performance, especially in terms of crown fire behavior, to identify potential model errors and establish model credibility.

In this paper, we develop an empirical relationship between the 10-m open wind speed and crown fire rate of spread using previously published field-scale data [28], and conduct a coarse assessment of crown fire rate of spread predictions from previously published FIRETEC and WFDS results. Given the limited information available in the empirical data set, we assessed model performance by examining the proportion of simulated predictions that fall within our empirically derived 95% prediction bands. We assumed that reasonable model performance was achieved based on criteria proposed by Rykiel [29] for dynamic ecological models. This approach is conceptually consistent with recommended standards for deterministic fire model evaluation in that we determined the degree to which ROS is an accurate representation of the real world from the perspective of the intended uses of ROS [30]. We then discuss challenges associated with assessment of detailed physics-based models, the potential sources of error in crown fire rate of spread measurements and predictions, and future research needs.

2 Overview of Detailed Physics-Based Modeling Approaches for Wildland Fire

Detailed physics-based models simulate fire behavior in a predictive manner by numerically solving, to an acceptable approximation, a coupled set of partial differential equations (PDEs). These PDEs govern the conservation of total mass, momentum, energy and chemical species. The intent of the set of coupled equations is to simulate the physical phenomena driving fire behavior, and their interactions, that control the behavior of a wildland fire. This approach allows for the evolution of various gas-phase quantities such as temperature, velocity and gaseous species mass fractions. Other quantities, such as smoke generation and transport can be predicted in three dimensions through time.

A large range of characteristic scales, both spatial and temporal, are relevant to turbulent combustion, conjugate heat transfer, and thermal degradation of the vegetation (i.e., drying, pyrolysis, and char oxidation). Detailed physics-based simulations of stand- and field-scale wildland fires are conducted with computational grids that are too coarse to directly resolve the small-scale processes or fine scale variability in the fuels complex (i.e., gaps between needles within a tree crown). Thus, the development of detailed physics-based modeling approaches must by necessity employ approximations to the governing equations to account for temporal and spatial scales that cannot be explicitly resolved. The numerical modeling of physical processes coupled to turbulent flow, on computational grids that are too coarse to resolve small-scale processes, has been an area of active research for decades [31, 32]. Although the modeling approaches in FIRETEC and WFDS differ in how they handle simplifications to the governing equations, both models use a finite-volume, large eddy simulation approach to model turbulence, where the large-scale eddies are explicitly resolved in numerical grids and small eddies are simulated with sub-grid scale models. The fuel complexes in both of these models are described as a highly-porous medium within the 3D numerical grids and are characterized by mean or bulk quantities (e.g. surface area to volume ratio, moisture content, and bulk density) of the thermally-thin vegetation components of the overall fuel complex. Because FIRETEC and WFDS were primarily developed to predict the evolution of the flaming front, they require the spatial and thermo-physical characteristics of the thermally thin component of the vegetative fuel; non-thermally thin fuels are assumed to not significantly contribute to the flaming front

Both FIRETEC and WFDS are based on the physical principals of conservation of mass, momentum and energy and use finite-volume, large-eddy approximations in space and time to predict the relevant dependent variables. Unresolved quantities are parameterized through closure schemes. The dependent variables reside at grid points that represent adjacent hexahedral volumes referred to as grid cells. The union of all grid cells determines the domain volume that is modeled. Although FIRETEC and WFDS both have similar capabilities there are subtitle differences in their solution techniques and parameterizations.

FIRETEC is primarily based on a fully explicit, compressible, atmospheric dynamics model. Because this model was designed for high-resolution simulation of atmospheric flows, the appropriate conservation of energy equation is posed in terms of potential temperature, a conserved quantity closely related to temperature [1, 15]. The turbulence model implemented in FIRETEC was developed by Linn [33]. This approach uses transport equations for turbulent kinetic energy at multiple specified length scales with a Boussinesq approximation to estimate the Reynolds stresses associated with these length scales. The Reynolds stress terms are used to close the momentum equations. As mentioned above FIRETEC represents vegetative fuels as porous media, which are described using bulk properties. Combustion is represented through a mixing-limited single-step model. This model requires sufficient quantities of heat, fuel and oxygen for fire to manifest. This formulation recognizes that for numerical cells at coarse scales the mean temperature is not adequate for determining if there is some material in the cell that is hot enough to burn. A probability distribution function of temperature accounts for the fraction of fuel in the cell that is of a high enough temperature to burn. The radiation model in FIRETEC is a Monte-Carlo algorithm that emits packets of photons in a random direction until the relative spatial distribution of the absorbed energy emitted at a given point in time is converged. The emitted energy from each hot cell is deposited according to the distribution. For more details on the description of FIRETEC, the reader is referred to [1, 15, 21, 22, 33, 34].

WFDS [20] is an extension of the Fire Dynamic Simulator (FDS), a fire engineering model developed at the National Institute of Standards and technology [35]. WFDS can utilize either a direct numerical simulation or a large-eddy approximation for the solution of the conservation equations. Unlike FIRETEC, the energy equation in WFDS is posed in terms of enthalpy rather than potential temperature. WFDS uses a low Mach-number approximation, which allows for thermally driven flow without the small time step restriction of a fully compressible model. With the low Mach number approximation, the pressure field is determined by formulating a Poisson equation from the momentum equations. The use of a low Mach-number approximation represents one the major differences between the two models. WFDS has a variety of LES turbulence schemes to choose from. For these simulations presented in this manuscript the Deardorff turbulence model [36] was used. The wildland fuels in WFDS are modeled as a porous media described by bulk characteristics in a similar fashion to FIRETEC. However, thermal degradation of solid fuel is modeled as a two step process where drying of the solid fuel occurs first followed by pyrolysis [20]. The gas phase combustion model in WFDS is also simulated as a mixing limited model; however in WFDS the temperature field within each grid-cell is obtained using the equation of state. Differences in the pyrolysis and combustion models between WFDS and FIRETEC represent another significant point of divergence among the two models. The radiation package determines the heat flux with a finite-volume scheme that decomposes solid angles in spherical coordinates with symmetry about the source. Additional details of WFDS model formulation are found in [2, 20, 35].

To date, both FIRETEC and WFDS have had a limited amount of model assessment in part due to a lack of field-scale experimentation or observation in which data collection is adequately complete for detailed physics-based model evaluation. Detailed information about the verification and validation of WFDS and FDS (the parent model of WFDS) can be found in [37, 38], whereas specific evaluation studies of WFDS for vegetative fuels can be found in [2, 20, 39–43]. Comparisons between FIRETEC and observations or field-scale data can be found in [1, 26, 34, 44, 45]. However, to date there have been very few comparisons between either of these models and measured crown fire rates of spread; Linn et al. [46] and Hoffman et al. [26] are exceptions.

3 Methods

3.1 Empirical Crown Fire Database

Alexander and Cruz [28], hereafter referred to as AC06, compiled a total of 57 wildfire observations from North American forests. The data set consisted of 43 fires from Canada, primarily occurring in boreal forest fuel types and 14 fires from the United States that occurred in pine dominated fuel types in the interior Rocky Mountains, the Lake States and the southeastern U.S (Table 1). All fires were categorized as either active or passive crown fires in AC06. During the development of this data set, any case study that occurred in areas with a mix of fuel types that do not support crown fire or occurred in areas with complex topography (>10% slope, or cross slope fire spread), were removed [28]. For the remaining observations, they reported the major fuel type, the ambient temperature (°C), the relative humidity (%), the effective fine fuel moisture (%), the canopy bulk density (kg m−3) (CBD) and the 10 m open wind speed (km h−1). Although AC06 compiled a relatively large amount of detailed information regarding the rate of fire spread during crown fires, several calculations were performed to modify the data set to meet their purpose. Specifically, they: (1) increased the 6.1-m open wind speed for all data from the U.S. by 15% to approximate the 10-m open wind speed, (2) assumed a constant CBD for a given forest type and thus did not consider the actual fuel loads or the horizontal and vertical distribution of fuels, and (3) estimated the effective fine-fuel moisture content using equations published by Rothermel [5], assuming that all fuels were shaded from solar radiation. Despite the need for such assumptions and the use of estimated values and additional calculations, AC06 remains the largest empirical data set assembled to date on crown fire rate of spread (ROS).

Table 1 Empirical Data Set Compiled from AC06 [28]
Full size table

3.2 Crown Fire ROS Predictions from FIRETEC and WFDS

Using a combination of online database searches and personal communications, we identified simulations conducted with WFDS and FIRETEC that: (1) were in North American and European forest types, (2) reported both the 10-m open wind speed and the crown fire ROS, (3) had at least 25% crown fuel consumption, and (4) simulated fire spread over at least 2 ha. We identified a total of 66 simulations, 38 of which were conducted using FIRETEC and 32 of which were conducted using WFDS. Identified simulations were from a combination of peer-reviewed publications and non-peer reviewed reports including conference proceedings and theses/dissertations (Table 2). Unlike the AC06 dataset the simulated fuels complexes were based on field measurements and published tree level biomass equations and include to some extent the vertical and horizontal distribution of canopy fuels at the measured sites. The details of the scenarios being modeled and the configuration of the simulations as well as a richer set of results can be found in each of the respective manuscripts. It is important to note that none of the simulations conducted in these studies were developed using data from the AC06 data set and instead relied on a combination of field measurements, published values and user assumptions to initialize the simulations. In no cases, was any form of calibration performed using the AC06 data sets, thus making these data a useful set of values to compare against empirical data set of AC06.

Table 2 Source of the Physics-Based Modeling Data Used in Comparisons
Full size table

3.3 Analysis

Because the AC06 data did not contain data needed for detailed physics-based model initialization, point- to-point comparisons between model predictions and the empirical data reported by AC06 were not possible. Instead, we used linear regression to assess the relationship between the 10-m open wind speed and the crown fire ROS for the data set reported by AC06 and estimated non-simultaneous 95% prediction intervals for a new observation. Since the other variables contained in the AC06 data set (i.e. canopy bulk density and effective fine-fuel moisture) were estimated by AC06 without knowledge of the actual conditions rather than directly measured or inferred from field collected data, we only included the 10-m open wind speed in our final regression (Eq. 1) and we have not included vertical canopy distribution or moisture levels in this regression.

$$ CROS = 24.5 + (0.669*10\;U) + (0.0373*10\;U^{2} ) $$
(1)

where CROS is the crown fire rate of spread and 10 U is the open 10-m above tree height wind velocity. The overall R2 value of our regression was 0.56. We then compared the simulated crown fire ROS from FIRETEC and WFDS to the 95% prediction intervals and assessed the number of points for each model that fell within these intervals. Based on an initial proposal for assessing model performance for dynamic ecological models [29], we assumed reasonable agreement between the simulations and the experimental data would occur if at least 75% of the simulated points were within the 95% predictions bounds.

4 Results

Figure 1 compares the simulated crown fire rates of spread from both FIRETEC and WFDS to the AC06 data. The AC06 data is represented by gray crosses; black triangles represent ROS simulated with FIRETEC, and the black circles represent ROS simulated with WFDS. Overall, 59% of 69%, or 85.5% of all simulated ROS values fell within the 95% prediction interval. Of the 10 crown fire ROS predictions that were outside the 95% predictive intervals, four were from WFDS and six were from FIRETEC. This corresponded to 87.5% and 84.2% of the predicted crown fire rates of spread falling inside of our predictive intervals for WFDS and FIRETEC respectively (Table 3; Figure. 1). All ten points that fell outside the 95% predictive intervals had greater rates of spread than expected for the given open wind speeds suggesting the potential for over-prediction from both WFDS and FIRETEC (Table 3).

Figure 1
figure 1

Crown fire ROS curve (black line) as a function of open 10-m wind speed with 95% confidence intervals (gray lines), determined from the data set of AC06 [28]. Points represent crown fires from: AC06 data set (gray crosses), FIRETEC (black triangles), and WFDS (black circles)

Full size image
Table 3 Overview of WFDS and FIRETEC Simulations Used in Comparisons
Full size table

Further analysis of the points that fell outside the predictive bounds was limited due to a lack of information regarding many of the independent variables known to influence crown fire behavior. However, four of the six FIRETEC points that showed an over-prediction bias were from a study conducted by Linn et al. [22]. In this study, the authors represented the initial atmospheric boundary conditions as a constant no shear wind profile. Comparisons of the normalized wind profile in these simulations to measured and theoretical profiles suggests that this assumption resulted in higher near-surface and within-canopy wind speeds than would be expected and could explain the increased rate of spread predictions in these cases. Mueller et al. [40] recently demonstrated that the use of a forced profile was not appropriate for accurate wind simulations above the forest and has not been used in recent FIRETEC publications. The remaining two FIRETEC points and the four WFDS points that were outside our predictive bounds were simulated to represent a piñon (Pinus edulis)/juniper (Juniperus spp.) woodland, a black spruce (Picea mariana) forest and a ponderosa pine (Pinus ponderosa) forest. In all these cases, except for the FIRETEC point that represented a black-spruce forest, the canopy bulk densities used in the simulations were greater than those reported by AC06, which might explain the high rates of predicted crown fire spread. For example the three WFDS simulations had simulated canopy bulk densities that ranged from 0.21 to 0.25 kg m−3, while the FIRETEC piñon/juniper woodland simulation had a mean canopy bulk density of 0.25 kg m−3. In contrast, AC06 used a canopy bulk density value of 0.13 to represent ponderosa pine dominated forests and had a maximum canopy bulk density value of 0.20 kg m−3 to represent other forest types. However, these potential differences should be interpreted with caution since the reported AC06 values were not based on actual on-site data and thus may not provide an accurate estimate of the actual canopy bulk densities of these fires.

5 Discussion

Our findings show that, generally speaking, both FIRETEC and WFDS produced crown fire rate of spread predictions within the range of a large empirical crown fire dataset. More specifically, over 80% of the FIRETEC and WFDS predictions of crown fire rate of spread fell within the 95% prediction bands of crown fire rate of spread as predicted by 10-m open wind speed. As noted by Cruz and Alexander [77], there are no generally agreed upon criteria for assessing wildland fire behavior modeling performance. However, Rykiel [26] suggested that in cases without explicit performance criteria the best that can be done is to state the model performance criteria and leave it to the end user to judge if these criteria are adequate for a given purpose. In the current study, our ability to test the model was limited by a lack of environmental and fuels data in AC06 such that we were not able to estimate the errors associated with point-to-point or concurrent comparisons. However, as suggested by Rykiel [26], an appropriate convention for performance criteria may include not only how close to observed data the simulation is, but also how often. Based on his initial recommendations for criteria in dynamic ecological modeling that model predictions should fall within the 95% prediction bounds 75% of the time, the two detailed physics-based models we tested provide reasonable agreement with the experimental data. In cases where WFDS and FIRETEC did not predict rates of spread within our predictive bounds, both models appear to over-predict the crown fire rate of spread. As suggested by Alexander and Cruz [28], under-predictions have severe implications for public and firefighter safety and fire operations planning; whereas over-predictions can be easily dealt with.

Although our comparisons provide some indication that detailed physics-based models produced reasonable estimates of crown fire rates of spread, these comparisons are significantly limited in scope for a variety of reasons. As indicated above, the comparisons made here rely on data from experiments that often have insufficient environmental and fuels information for a more complete evaluation of detailed physics-based models. The lack of detailed information had three important outcomes. First, we were not able to directly simulate experiments reported by AC06. This restricted our evaluation to simple statistical comparisons based on the development of 95% prediction bounds around the relationship between the measured 10-m wind velocity and the crown fire rates of spread. While this allowed us to determine how often simulated results were similar to the observations, it did not allow us to quantify the error associated with the predictions, if the model produces outputs with the same statistical properties of the observational data set or to assess the simulated relationships between other fuels and environmental variables and crown fire ROS.

Secondly, insufficient data limited our ability to further evaluate any discrepancies between the model simulations and our expectations based on our prediction bounds. In particular, many of the experiments contained within the AC06 data set lack sufficient information regarding the spatial and temporal variability of the fuels complex and the atmospheric boundary layer to fully assess discrepancies. For example, often an investigator will provide a single number when describing the wind speed near the fire or the fuels complex, despite both wind speed magnitude and canopy fuel loading having a strong dependence on the height above ground and significant horizontal variability. Recent numerical simulations have shown that rate of spread predictions from detailed physics-based models are sensitive to small variations in both the spatial pattern of the fuels complex [22, 23] and assumptions regarding the atmosphere boundary layer [46]. Clearly, further assessment of detailed physics-based models would benefit from additional data regarding spatial and temporal variability of key fuel and environmental characteristics (i.e. wind). However, new approaches may have to be developed to make such data collection logistically possible at the scales over which crown fire spread occurs. For example, quantifying the pattern of surface and canopy fuel loading could potentially be characterized using both active and passive remote sensing technologies. However, there are still limitations within the temporal and spatial availability of these technologies.

Finally, the only fire behavior data reported in the AC06 data set was the head-fire rate of spread which is the end result of many coupled physical processes. Ideally, detailed physics-based models should be validated at both the sub-model level and the system level. Assessment of both system and sub-level model behavior are important as they can provide checks against incorrect conclusions, indicate if the model is conceptually consistent with reality, and identify specific components of the model that need to be improved [78].

In addition, insufficient data limiting assessment of model performance, the comparisons made here assumed that the field-collected data are without error. In reality, there are inaccuracies and imprecisions of both the dependent and independent data collected during field-scale wildland fire experiments, which place additional limits on the level of precision and accuracy that can be expected from a model. As suggested by Belloccie et al. [78], the accuracy of a model is determined in part by the ability of the model equations to describe the real world and in part by the quality of both the input data used to initialize the model and the dependent parameter data used to evaluate model outputs. In many cases, the errors associated with both the estimated input parameters used to initialize detailed physics-based models and the dependent output parameters are rarely reported and often are unknown. Although the uncertainty in measured data used to assess model performance is frequently acknowledged [78, 79], it is rarely considered in model performance evaluation.

Although the empirical data from past experiments is useful for coarse-scale comparisons, such as those made here, it is often insufficient for a more complete assessment of detailed physics-based models in part due to the reasons discussed above. The lack of empirical data that has been collected with the intention to assess detailed physics-based models is currently acting as a bottleneck for further assessment of model performance and development. In addition, since detailed physics-based models are often used for investigation of phenomenology or hypothesis development regarding cause and effect relationships, it is important that future studies strive to provide better and more inclusive initialization parameter estimates, including the temporal and spatial dynamics associated with the wind field and fuels complex, but also evaluate a broader range of fire behavior measures beyond just the head-fire rate of spread. For example, there is little information available regarding the heterogeneous nature of the rate of fire spread, rate of mass consumption, heat fluxes or fire depth through time for crown or surface fires at field-scales. Although there have been a number of model performance assessments of both FIRETEC and WFDS, additional experimentation that is specifically designed to assess a broader array of detailed physics-based model outputs combined with a wider range of statistical analysis of model performance could help identify any potential errors and limitations, enhance model credibility, and expand the robustness of model assessment and model application.