Abstract
The paper deals with the validation and evaluation of mathematical models in natural hazard analysis, with a special focus on establishing their predictive power. Although most of the tools and statistics available are common to general classification models, some peculiarities arise in the case of hazard assessment. This is due to the fact that the target for validation, the propensity to develop a dangerous characteristic, is not really known and must be estimated from a (usually) very small sample. This implies that the two types of errors (false positives and false negatives) should be given different meanings. Related to this, a very frequent situation is the presence of prevalence (different proportion of positive and negative cases) in the sample. It is shown that sample prevalence can have a dramatic effect in some very common validation statistics, like the confusion matrix and model efficiency. Here some statistics based on the confusion matrix are presented and discussed, and the use of threshold-independent approaches (especially the ROC plot) is shown. The ROC plot is also proposed as a convenient tool for decision-taking in a risk management context. A general scheme for hazard predictive modeling is finally proposed.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
B. P. Bledsoe C. C. Watson (2001) ArticleTitleLogistic analysis of channel pattern thresholds: meandering, braiding, and incising Geomorphology 38 IssueID3–4 281–300
A. Carrara (1983) ArticleTitleMultivariate models for landslide hazard evaluation Math. Geol. 15 IssueID3 403–426
A. Carrara M. Cardinali R. Detti F. Guzzetti V. Pasqui P. Reichenbach (1991) ArticleTitleGIS techniques and statistical models in evaluating landslide hazard Earth Surf. Process. Landforms 16 427–445
C. F. Chung A. G. Fabbri (1999) ArticleTitleProbabilistic prediction models for landslide hazard mapping Photogram. Eng. Remote Sensing 65 IssueID12 1389–1399
C. F. Chung A. G. Fabbri (2003) ArticleTitleValidation of spatial prediction models for landslide hazard mapping Nat. Hazards 30 451–472 Occurrence Handle10.1023/B:NHAZ.0000007172.62651.2b
Chung, C. F., Fabbri, A., and Van Westen, C. J.: 1995. Multivariate regression analysis for landslide hazard zonation. In A. Carrara & F. Guzzetti (Eds.), The Nederlands: Kluwer Academic Publishers (pp. 107–133).
A. Clerici S. Parego C. Tellini P. Vescovi (2002) ArticleTitleA procedure for landslide susceptibility zonation by the conditional analysis method Geomorphology 48 349–364 Occurrence Handle10.1016/S0169-555X(02)00079-X
F. C. Dai C. F. Lee (2002) ArticleTitleLandslide characteristics and slope instability modeling using GIS, Lantau Island, Hong Kong Geomorphology 42 IssueID3–4 213–228
Deleo, J. M.: 1993, Receiver operating characteristic laboratory (ROCLAB): software for developing decision strategies that account for uncertainty. In: Proceedings of the Second International Symposium on Uncertainty Modelling and Analysis, Computer Society Press, College Park, pp. 318–325
A. H. Fielding J. F. Bell (1997) ArticleTitleA review of methods for the assessment of prediction errors in conservation presence/absence models Environ. Conserv. 24 IssueID1 38–49
J. A. Floyer D. M. McClung (2003) ArticleTitleNumerical avalanche prediction: Bear Pass, British Columbia, Canada Cold Regions Sci. Technol. 37 IssueID3 333–342
A. D. Forbes (1995) ArticleTitleClassification-algorithm evaluation: five performance measures based on confusion matrices J. Clin. Monitor. 11 IssueID3 189–206
D. J. Furbish R. M. Rice (1983) ArticleTitlePredicting landslides related to clearcut logging, Northwestern California, USA Mount. Res. Dev. 3 IssueID3 253–259
S. Lee J. Choi K. Min (2002) ArticleTitleLandslide susceptibility analysis and verification using the Bayesian probability model Environ. Geol. 43 120–131
A. Lorente J. M. García-Ruiz S. Beguería J. Arnáez (2002) ArticleTitleFactors explaining the spatial distribution of hillslope debris flows: a case study in the Flysch Sector of the Central Spanish Pyrenees Mount. Res. Dev. 22 IssueID1 32–39
J. A. Martínez-Casasnovas M. C. Ramos J. Poesen (2003) ArticleTitleAssessment of sidewall erosion in large gullies using multi-temporal DEMs and logistic regression analysis Geomorphology 58 305–321
D. D. Massie K. D. White S. F. Daly (2002) ArticleTitleApplication of neural networks to predict ice jam occurrence Cold Regions Sci. Technol. 35 IssueID2 115–122
R. P. C. Morgan D. Mngomezulu (2003) ArticleTitleThreshold conditions for initiation of valley-side gullies in the Middle Veld of Swaziland Catena 50 IssueID2–4 401–414
H. Neuland (1976) ArticleTitleA prediction model of landslips Catena 3 215–230 Occurrence Handle10.1016/0341-8162(76)90011-4
F. V. Perry G. A. Valentine E. K. Desmarais G. WoldeGabriel (2001) ArticleTitleProbabilistic assessment of volcanic hazard to radioactive waste repositories in Japan: intersection by a dike from a nearby composite volcano Geology 29 IssueID3 255–258 Occurrence Handle10.1130/0091-7613(2001)029<0255:PAOVHT>2.0.CO;2
J. Remondo A. González-Díez J. R. D. D. Terán A. Cendrero (2003) ArticleTitleLandslide susceptibility models utilising spatial data analysis techniques. A case study from the Lower Deba Valley, Guipuzcoa (Spain) Nat. Hazards 30 IssueID4 233–249
Rice, R. M. and Pillsbury, N. H.: 1982, Predicting landslides in clearcut patches. Symposium on Recent Development in the Explanation and Prediction of Erosion and Sediment Yield. International Association of Hydrological Sciences, pp. 303–311
D. N. Rowbotham D. Dudycha (1998) ArticleTitleGIS modelling of slope stability in Phewa Tal watershed, Nepal Geomorphology 26 IssueID1–3 151–170
R. Beek ParticleVan T. Asch ParticleVan (2004) ArticleTitleRegional assessment of the effects of land-use change on landslide hazard by means of physically based modelling Nat. Hazards 31 289–304
C. J. Westen ParticleVan N. Rengers M. T. J. Terlien R. Soeters (1997) ArticleTitlePrediction of the occurrence of slope instability phenomenal through GIS-based hazard zonation Geol. Runds. (Int. J. Earth Sci.) 86 IssueID2 404–414
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Beguería, S. Validation and Evaluation of Predictive Models in Hazard Assessment and Risk Management. Nat Hazards 37, 315–329 (2006). https://doi.org/10.1007/s11069-005-5182-6
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11069-005-5182-6