Benchmarking Initiatives

Abstract

Benchmarking has been recognized as an efficient tool for model validation as well as scientific collaboration. Several workshop series have been established to foster the benchmarking idea: setting up test cases with increasing complexity for method development and code comparison. In addition to representing model complexity, one of the key efforts is to develop codes that are suitable for modern HPC platforms such as PetaFlop supercomputers. It has been realized that those challenges are beyond single team capabilities. Some of these ongoing initiatives are, for example, Sim-SEQ (Mukhopadhyay et al. 2012, 2015), CO2BENCH (for CO2 storage) (Kolditz et al. 2012a), DECOVALEX (1.1), SeS-Bench (1.2) (Steefel et al. 2014) and MoMaS (1.3) (Ackerer 2010).

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Benchmarking has been recognized as an efficient tool for model validation as well as scientific collaboration. Several workshop series have been established to foster the benchmarking idea: setting up test cases with increasing complexity for method development and code comparison. In addition to representing model complexity, one of the key efforts is to develop codes that are suitable for modern HPC platforms such as PetaFlop supercomputers. It has been realized that those challenges are beyond single team capabilities. Some of these ongoing initiatives are, for example , Sim-SEQ (Mukhopadhyay et al. 2012, 2015), CO2BENCH (for CO2 storage) (Kolditz et al. 2012a), DECOVALEX (1.1), SeS-Bench (1.2) (Steefel et al. 2014) and MoMaS (1.3) (Ackerer 2010).

1 DECOVALEX

Jens Birkholzer

DECOVALEX is a multinational model comparison project for advancing the understanding and mathematical modeling of coupled thermo-hydro-mechanical (THM) and thermo-hydro-chemical (THC) processes in geologic and engineered systems associated with geologic disposal of radioactive waste. DECOVALEX is an acronym for “Development of Coupled Models and their Validation against Experiments” Starting in 1992, the project has made important progress and played a key role in the development and validation of advanced numerical models. In-depth knowledge has been gained of the complex THM and THC behavior of different host rock formations and buffer/backfill materials, and significant advances have been made in numerical simulation methods for their quantitative analysis. Over the years, DECOVALEX has involved research teams from a large number of radioactive-waste-management organizations and regulatory authorities, from countries such as Canada, Czech Republic, China, Finland, France, Germany, Japan, Republic of Korea, Spain, Sweden, Switzerland, United Kingdom, and the United States. A good overview is provided at www.decovalex.org.

1.1 DECOVALEX Framework

DECOVALEX activities are typically conducted in separate four-year modeling phases. Each phase features a small number (typically three to six) of modeling tasks of importance to radioactive waste disposal. Modeling tasks can either be Test Cases (TC) or Benchmark Tests (BMT). TCs are laboratory and field experiments that have been conducted by one of the project partners and are then collectively studied and modeled by DECOVALEX participants. BMTs involve less complex modeling problems, often targeted at comparing specific solution methods or developing new constitutive relationships. Participating research teams work collaboratively on selected modeling cases, followed by comparison of model results between different models and between models and measurements. While code verification and benchmarking efforts have been undertaken elsewhere to test simulation codes, the model comparison conducted within the DECOVALEX framework is different, because (a) the modeling tasks are often actual laboratory and field experiments, and (b) DECOVALEX engages model comparison in a broad and comprehensive sense, including the modelers’ choice of interpretation of experimental data, boundary conditions, rock and fluid properties, etc., in addition to their choice of simulators. Over the years, a number of large-scale, multiyear field experiments have been studied within the project (e.g., the Kamaishi THM Experiment in Japan, the FEBEX heater test at Grimsel Test Site in Switzerland, and the Yucca Mountain Drift-Scale Heater Test).

1.2 Current DECOVALEX Activities

DECOVALEX-2015 is the current and 6th project phase and runs from 2012 through 2015. Modeling teams from ten international partner organizations participate in the comparative evaluation of five modeling tasks involving complex field and/or laboratory experiments in Switzerland, France, Japan, and Czech Republic:

• SEALEX Experiment: HM Simulation of a long-term test of the hydraulic (sealing) performance of a swelling bentonite core (5 m long) in a mini tunnel (60 cm diameter) at the Tournemire Underground Research Laboratory (URL) in France,

• HE-E Heater Test: THM Simulation of bentonite/rock interaction to evaluate sealing and clay barrier performance, in a micro-tunnel at the Mont Terri URL in Switzerland,

• EBS Experiment: Simulation of the THMC behavior of the EBS under heating conditions in both the early resaturation and post-closure stages of the repository, in a vertical emplacement hole at the Horonobe URL in Japan (see Fig. 1.1),

• Bedrichov Tunnel Experiment: Model interpretation of inflow patterns and tracer transport behavior in a fractured granite formation in the Czech Republic.

Together, these tasks address a wide range of relevant issues related to engineered and natural system behavior in argillaceous and crystalline host rocks. A new DECOVALEX phase will start in April 2016 and run until December 2019, referred to as DECOVALEX-2019.

Fig. 1.1

EBS Experiment at Horonobe URL: design of monitoring boreholes for sensor installation (from DECOVALEX web site, www.decovalex.org)

1.3 DECOVALEX Success Story

The +20-year “young” DECOVALEX project has played, and continues to play, a key role in the development of the coupled THM, THC, and recently THMC, models of geosystems and their applications to deep geological disposal of radioactive waste and spent nuclear fuel. New and sophisticated coupled simulation tools have been developed to study the near- and far-field behaviors of potential nuclear waste repositories under various scenarios, including understanding geosphere responses to glaciation and permafrost. A model comparison framework has been developed that fosters intensive interactions among research teams of different disciplines and modeling approaches, and that features successful application to both complex in situ experiments and benchmark test problems. Many doctoral theses were completed in the course of this project and related model results and comparisons have been published in Special Issues in different journals, several other journal contributions, book chapters, and progress reports. The results of these efforts have been of major benefit to the advancement of knowledge on waste disposal in general, and to funding organizations of the DECOVALEX project in particular. The research teams have been able to share data and results from very expensive long-term and large-scale field tests, generate new ideas and concepts, raise technical issues in joint publications, and perform critical reviews of each other’s work. The insight obtained in such integrated, cooperative efforts would have been impossible if the teams had worked independently.

Special Issues

• Special Issue on DECOVALEX 2011—Part 1, Journal of Rock Mechanics and Geotechnical Engineering (Volume 5, Issue 1, Pages 1–84, 2013),

• Special Issue on DECOVALEX 2011 Part 2, Journal of Rock Mechanics and Geotechnical Engineering (Volume 5, Issue 2, Pages 85–155, 2013),

• The DECOVALEX-THMC PROJECT (Safety assessment of nuclear waste repositories), Journal of Environmental Geology (Volume 57, Number 6, Pages 1217–1456, 2009),

• Special Issue on Research Results from the DECOVALEX III & Benchpar projects, International Journal of Rock Mechanics and Mining Sciences (Volume 42, Number 5–6, Pages 591–870, 2005),

• Special Issue in Elsevier Geo-Engineering Book Series: Coupled Thermo-Hydro-Mechanical-Chemical Processes in Geo-Systems—Fundamentals, Modelling, Experiments and Applications (Volume 2, Pages 3–832, 2004).

• DECOVALEX-II Special Issue, International Journal of Rock Mechanics and Mining Sciences (Volume 38, Number 1, Pages 1–162, 2001).

• Special Issue in Elsevier Book Series Developments in Geotechnical Engineering: Coupled Thermo-Hydro-Mechanical Processes of Fractured Media: Mathematical and Experimental Studies (Volume 79, Pages 1–575, 1996).

• The results of the ongoing DECOVALEX phase will be published in a Thematic Issue of Environmental Earth Sciences (EES) in 2016, forthcoming.

2 Subsurface Environmental Simulation Benchmarking (SeS-Bench)

Steve Yabusaki

The Subsurface Environmental Simulation Benchmarking (SeS-Bench ) working group was initiated to develop, publish, and make publicly available well-described benchmark problems that could be used to demonstrate simulator conformance with norms established by the subsurface science and engineering community. The working group held its first organizational meeting in Berkeley, California, USA in 2011 with follow-up workshops at the National Central University in Taipei, Taiwan; the Helmholtz Centre for Environmental Research in Leipzig, Germany; and in Cadarache, France, co-organized by the Commisariat $\grave{a}$ l’Energie Atomique (CEA), ANDRA, BRGM and MINES ParisTech (Fig. 1.2). The activities of the SeS-Bench working group have led to the development and publication of a special issue of the journal, Computational Geosciences,¹ “Reactive Transport Benchmarks for Subsurface Environmental Simulation” in 2015.

SeS-Bench supports the development of environmentally-relevant benchmark problem sets that provide rigorous tests for process model conceptualization, numerical implementation, process coupling, and accuracy. The scope of the benchmarking targets simulation capabilities based on the continuum (non-pore-scale) modeling approach for porous media. Each benchmark contribution consists of a principal comprehensive problem and supporting component problems. This is intended to isolate key attributes of the benchmark problem for higher scrutiny before addressing the full complement of interacting processes, properties, and conditions. To ensure the accuracy and validity of a published benchmark, SeS-Bench requires each benchmark to have a minimum of three different code bases to obtain the same or nearly the same results. Workshops are used as a forum to propose benchmark problem sets, recruit participants (and simulators) for proposed benchmarks, and report on findings and progress.

The initial SeS-Bench focus on multicomponent reactive transport modeling reflects the expanding range of mechanisms and increasing complexity needed to address issues relevant to climate change, nuclear waste management, contaminant remediation, and pollution prevention. The published reactive transport modeling benchmarks, while addressing comprehensive and detailed biogeochemical reaction networks, generally have relatively simple flow fields and transport specifications. The intent is for these benchmarks to be accessible to a wide range of interests and skill levels. All the benchmark problem sets contain specifications that can be run using modest standard computing resources. Furthermore, the simpler component benchmarks included in each problem set require less expertise and time to address. The belief is that these benchmark problem sets can also be instructional, providing a progression from conceptualization to the specification of actual reaction networks. Even for experienced modelers, these benchmarks can provide an entree into new reaction processes and provide templates that can be adapted for other scenarios. The hope is that this benchmarking effort will aid the training of a new generation of modelers who will continue to expand the role of reactive transport modeling in subsurface environmental simulation.

Fig. 1.2

SeS-Bench Meeting 2014 in Cadarache (Photo by Olaf Kolditz)

The 14 contributed articles in the Computational Geosciences special issue are from teams of participants in the SeS-Bench working group. A high level and non-exhaustive overview of the mathematical and numerical bases of modern reactive simulators is provided in the first paper. The benchmark problems are organized by topic: (1) microbially-mediated reactions, (2) isotopes, (3) multi-component diffusion, (4) uranium fate and transport, (5) metal mobility in mining affected systems, and (6) geologic waste repository material interactions.

Microbially-Mediated Reactions. Microbially-mediated Cr reduction in a one-dimensional column experiment was investigated with standard Monod (or Michaelis-Menten) rate formulations using sediment and chemical conditions from a contaminated site at the Hanford Nuclear Reservation in Washington State, USA. The remarkable agreement between the codes on this benchmark likely reflects in part the focus on kinetic as opposed to thermodynamic formulations, which proved to be the biggest source of discrepancies in many of the benchmarks.

Isotopes. A benchmark problem was developed for the simulation of kinetic Cr isotope fractionation in 1D and 2D domains. The benchmark was based on a field study where Cr(VI) reduction and accompanying Cr isotope fractionation occurs abiotically through reaction with dissolved Fe(II). The benchmark demonstrates that excellent agreement can be obtained between codes, despite differences in the approaches to implement isotope fractionation.

Multi-component Diffusion. The number of reactive transport simulators that include electrochemical migration and consider the chemical potential gradient as a driving force of diffusion has been growing in recent years. Two multi-component diffusion benchmarks used the Nernst-Planck equation rather than the simpler Fick’s Law to account for electrochemical migration. The first benchmark problem set was specifically designed to highlight strong electric coupling effects. A second benchmark addressed multi-species diffusion through compacted bentonite and included an explicit treatment of electrostatic effects associated with charged clay surfaces.

Uranium Fate and Transport. Uranium mobility is notoriously sensitive to pH, Eh, alkalinity, major ions, surface sites, etc. The three benchmarks that considered uranium fate and transport represent some of the successes in using coupled process modeling to develop a systematic understanding of uranium behavior in naturally complex subsurface materials. The first benchmark was based on a field experiment conducted at the Rifle site in western Colorado, USA in which acetate was injected into uranium-contaminated groundwater to drive microbially-mediated reduction and immobilization of uranium. A second benchmark focused on the important role of Fe-hydroxides in re-oxidizing biogenically reduced uranium. A third benchmark focused on multi-rate models to simulate uranium leaching and migration.

Metal Mobility in Mining Affected Areas. Two benchmarks considered metal mobility in the context of mining activities. The first used reactive transport modeling to simulate the generation of acidity as a result of sulfide oxidation and its subsequent effect on metal mobility above and below the water table. A second benchmark focused on metal accumulation and mobility in lake sediments downstream of mining operations. The modeling assumed a 1-D geometry that extends from the lake-sediment interface to depth and includes component problems with and without sediment burial.

Geologic Waste Repository Material Interactions. Three benchmarks focused on topics related to the storage of nuclear waste in geological repositories. The first addressed reactive transport associated with large concentration gradients across a cement-clay interface, as might be expected in the engineered barrier system for a waste repository. The challenge in this problem resulted from the high geochemical contrast between the cement and clay, which drove diffusive mixing across the interface and nonlinear mineral dissolution and precipitation. A second benchmark relevant to nuclear waste repositories considered the effects of porosity, permeability and tortuosity evolution. The porosity reduction was the result of reactions (principally gypsum precipitation), which decreased the permeability to the point where flow was effectively stopped. The last benchmark focused on simulations of reactive transport and solute mobility resulting from cracks developed in concrete. The problem set addressed the effect of discrete fractures on the decalcification of cement structures.

3 MoMaS

MoMaS stands for Modeling, Mathematics and numerical Simulations and is related to nuclear waste management problems (Fig. 1.3). MoMaS proposes numerical benchmarks to help evaluate computer codes. MoMaS was initiated by a federation of French research groups involving CERMICS—Ecole Nationale des Ponts et Chaussees, Laboratoire Jacques-Louis Lions, UPMC Paris among others. Parkhurst and Wissmeier (2015) recently used MoMaS for testing a reaction module for transport simulators based on the geochemical model PHREEQC. MoMaS was extensively utilized for verifying a global reduction scheme for reactive transport modeling (RTM) (Amir and Kern 2010; Carrayrou et al. 2010b; de Dieuleveult and Erhel 2010; Hoffmann et al. 2010, 2012). Several RTM codes participated the MoMaS initiatives, e.g. HYTEC (Lagneau and van der Lee 2010), MIN3P (Mayer and MacQuarrie 2010), SPECY (Carrayrou 2010) and recently OpenGeoSys (Huang et al. 2015a). Carrayrou et al. (2010a) compared different numerical methods for simulating strongly nonlinear and heterogeneous reactive transport problems. A special issue in Computers and Geosciences was dedicated the results of the MoMaS benchmarking initiative (Ackerer 2010).

Fig. 1.3

The official MoMaS website and further information can be found at https://www.ljll.math.upmc.fr/cances/gdrmomas/benchmarks-en.html

3.1 Finished Benchmarks

MoMaS has a number of finished benchmarking exercises:

• Anisotropic and Heterogeneous Diffusion: A benchmark on numerical methods for heterogeneous and anisotropic diffusion in a 2D domain,

• Geochemistry: A benchmark on numerical aspects of coupling between transport and chemistry in porous media,

• Heterogeneous and anisotropic diffusion: A benchmark on numerical methods for heterogeneous and anisotropic diffusion,

• Couplex-Gas: A benchmark addressing the main problems to simulate two phases (liquid, gas) two components (water, H2) flows in porous media. This benchmark was defined by Andra, with a collaboration from MoMaS http://math.univ-lyon1.fr/homes-www/bourgeat/MoMaS/cas_test.

• Hydro-mechanical behavior of an excavation: A benchmark on geoscientific studies of the Excavation Damaged and Disturbed Zones in sedimentary rocks.

• Couplex: Several benchmarks are a set of realistic test cases aimed at simulating the transport of nuclides around a nuclear waste repository (Bourgeat et al. 2009).

3.2 Currently Running Benchmarks

MoMaS has two ongoing benchmarking exercises:

• Two phase flows: Several numerical test cases, with a simplified physics, so as to concentrate on a single numerical difficulty in each case (Marchand and Knabner 2014a)

• 3D diffusion: A benchmark addressing a three-dimensional anisotropic diffusion problem, which is discretized on general, possibly non-conforming meshes http://www.i2m.univ-amu.fr/latp_numerique/?q=node/4.

In this benchmark book new results are presented in Chap. 6 considering two-phase two-component transport processes with phase transition phenomenon related to the MoMaS Couplex benchmarks.