1 Introduction

Zeolites are porous crystalline materials that play a “cornerstone” role in various applications, such as catalysis, adsorption-based separations, and ion exchange (water softening) [1]. As a complement to experimental characterisation techniques, computational investigations have greatly contributed to an atomic-level understanding of these materials. A recent review by Van Speybroeck et al. [2] provides a comprehensive overview of the applications of computational chemistry techniques in zeolite science, covering both molecular mechanics and electronic structure calculations. In the latter field, density functional theory (DFT) plays a particularly prominent role, as it combines a high achievable accuracy with a moderate computational cost. On the one hand, the efficiency of DFT permits comparative studies of a large number of zeolite structures, exemplified by the DFT-based prediction of the elastic properties of 121 zeolite frameworks by Coudert [3]. On the other hand, the treatment of structures with very large unit cells is also possible. In previous work, we used DFT calculations to calculate system-specific charges for a variety of zeolites, the largest unit cell having a volume of more than 20,000 Å3 (primitive cell of PAU framework) [4].

It is well established that the result of a DFT calculation—and, thus, its potential usefulness—depends strongly on the choice of the exchange–correlation functional. As a consequence, the “benchmarking” of DFT calculations, i.e. the critical comparison of different functionals, has become a very active field of research. In such benchmarking studies, either experimental data or computational results from high-level wave-function-based methods (e.g. coupled-cluster calculations) are used as reference data. Depending on the quantity one aims to predict, most emphasis in the benchmarking can be placed on the accurate prediction of structures, energetics (thermochemistry), or other properties (e.g. vibrational spectra). An additional problem arises when dispersion interactions play a non-negligible role in the system of interest, as these interactions are not well represented in DFT calculations employing standard exchange–correlation functionals (such as the local density approximation (LDA), the generalised gradient approximation (GGA), or hybrid functionals) [5, 6]. To address this issue, different approaches of varying complexity have been proposed to incorporate dispersion interactions in DFT [5–8], leading to an additional degree of freedom in the setup of the computations. Benchmarking efforts in the field of organic non-covalent complexes, for which the accurate description of both intra- and intermolecular interactions is pivotal, have been presented, for example, by Grimme et al. [9, 10]. Other groups have focussed on the structure prediction of molecular crystals, where crystal packing effects play an important role [11–13]. To emphasise the wide variety of possibilities, it is worth noting that a single benchmarking study on small non-covalent dimers included a total of 382 different “flavours” of DFT (different exchange–correlation functionals with and without dispersion correction) [14].

Numerous researchers have investigated the performance of DFT for the prediction of structures and properties of inorganic crystalline solids. It has been established that LDA, the simplest approximation, tends to “overbind” structures, causing an underestimation of bond lengths and lattice parameters, whereas the opposite trend is observed for many GGA-type functionals (e.g. the PBE functional [15], which is particularly popular for solids) [16, 17]. While a complete discussion of previous DFT benchmarking efforts for inorganic crystals is beyond the scope of the present work, some results of particular relevance are summarised in the following. In order to discuss some of the most typical observations, the very recent study of Tran et al. [18] provides a good starting point. These authors compared the performance of a large portfolio of DFT approaches, comprising LDA, GGA, meta-GGA, and hybrid functionals, both with and without dispersion corrections, for lattice constants, bulk moduli, and cohesive energies of strongly bound and dispersion-dominated solids. For the strongly bound solids (elements and binary compounds, a total of 44 systems), some GGA-type functionals that were specifically designed for solids (such as the Wu-Cohen [WC] [19], PBEsol [20], and SG4 [21] functionals) provided results that were similarly accurate to those obtained with the best-performing meta-GGA and hybrid functionals. While the use of a dispersion correction did not always lead to better results, the authors observed a significant improvement in cases where the uncorrected functional had a clear tendency to overestimate the lattice parameters, e.g. for PBE. For systems in which dispersion interactions are important (rare gas crystals and layered solids, a total of 5 systems), none of the uncorrected functionals gave a satisfactory performance across the board, while the inclusion of a pairwise (Grimme-type) [22, 23] dispersion correction improved the results considerably.

A good performance of PBEsol and WC for the prediction of equilibrium structures was also observed in a series of comparative studies of different minerals (α-quartz, corundum, forsterite, pyrope, Al2SiO5 polymorphs) [24–26]. However, these functionals failed in the description of hydrogen-bonded layered aluminium hydroxides, where the length of hydrogen bonds was severely underestimated, and also for the case of brucite, Mg[OH]6, in which the layers are held together by dispersion interactions. For these systems, hybrid functionals, which incorporate a fraction of exact (Hartree–Fock) exchange and are thus computationally more demanding, were found to give better agreement with experiment [26]. A superior performance of these functionals was also found in a computational study of anhydrous and hydrated clinoptilolite, a natural zeolite [27]. Moreover, hybrids are better suited than GGA-type functionals when aiming at the computation of vibrational spectra [24–26] or band gaps [28].

In order to understand whether the accuracy can be improved by including a dispersion correction term, Tunega et al. [29] compared the performance of different GGA-type functionals (among them PBE and PBEsol), including some dispersion-corrected variants, in the prediction of the structural parameters of four clay minerals, where the layers are held together either by dispersion forces (talc, pyrophyllite) or by hydrogen bonds (lizardite, kaolinite). At variance with the previously discussed results on layered aluminium hydroxides and brucite, these authors observed a good performance of the PBEsol functional (without dispersion correction) for all four systems. Among the dispersion-corrected approaches, the PBE-TS functional [30] gave particularly good agreement with the experimental structures, while a moderate, but systematic, underestimation of the lattice parameters was found for PBE with the Grimme-type D2 correction [22]. Using hybrid functionals (PBE0 [31], B3LYP [32, 33]), Román-Román and Zicovich-Wilson [34] showed that the prediction of the relative stability of all-silica zeolites is improved by the inclusion of a dispersion correction. Interestingly, these authors observed a systematic overestimation of the molar volume (volume per SiO2 unit), which may exceed 10% for some zeolites even for the best-performing approach (PBE0-D2).

The large majority of previous studies of zeolites that compared the performance of different DFT approaches typically addressed only one or a few systems (e.g. an extensive benchmarking work addressing structure and various properties of Co- and Cu-exchanged chabazite [35–37], the previously mentioned study of clinoptilolite [27], and a very recent investigation of MFI nanosheets [38]) and/or focussed on the interaction of zeolite hosts with adsorbed molecules (e.g. water [39–41], NO [42], CO [43–45], CO2, CH4 and N2 [46], alkanes and/or alcohols [47–53]). While the study of Román-Román and Zicovich-Wilson [34] covers a total of 14 all-silica zeolites, these authors only reported the molar volumes, without discussing the structural parameters in more detail. In contrast, a benchmarking of DFT approaches that places most emphasis on the accurate reproduction of guest-free zeolite structures, considering a range of different topologies, has not yet been carried out. In order to fill this gap, the present study evaluates the performance of GGA-type functionals based on PBE in the crystal structure optimisation of neutral-framework zeotypes, more specifically, all-silica (SiO2) zeolites and aluminophosphate (AlPO4) zeotypes. As these materials do not contain any extra-framework cations (for which fractional occupancies are the rule, rather than the exception), and no mixed occupancies of the tetrahedral sites (which also occur very frequently in aluminosilicate zeolites), there is no necessity to modify the experimental structures in order to arrive at models that can be used in the DFT computations. Because the DFT-optimised structures are directly comparable to experimental crystal structure data, these neutral-framework zeotypes are particularly well suited for benchmarking purposes.

In addition to the original PBE functional and two PBE-based functionals that were specifically designed for solids (WC, PBEsol), two dispersion-corrected variants (PBE-D2 and PBE-TS) are included. As outlined above, WC and PBEsol perform well for dense solids, but less so for layered systems due to the poor description of the inter-layer interactions, where dispersion-corrected functionals are more accurate. Taken together, these findings do not provide clear guidelines for zeolites and related materials: while the three-dimensional connectedness of the framework indicates that WC/PBEsol should perform rather well for these systems, long-range interactions across the pores might also play a non-negligible role in determining the equilibrium structure, necessitating the inclusion of a dispersion correction term. The present selection of functionals allows us order to determine the relative importance of an accurate description of “short-range” and “long-range” effects. Although rather poor agreement with experiment can be expected for standard PBE, it is included as a reference functional to evaluate the relative improvement by using “post-PBE” approaches.

While it might be argued that a larger range of functionals should be included in the benchmarking, it should be reiterated that some of the functionals considered have been found to perform very well for related materials, such as sheet silicates [29]. Moreover, GGA-type functionals are particularly attractive due to their modest computational cost and may thus be employed for tasks where the use of more sophisticated functionals (meta-GGA, hybrids) is not (yet) feasible. To this end, the present study aims to elucidate how accurate results one can obtain with a relatively low-level DFT approach. Future work could then address whether the use of more elaborate functionals enables further improvements.

The remainder of this paper is organised as follows: after describing the choice of reference structures, the computational methods, and the means of error assessment, the results and discussion section begins with a detailed analysis of the DFT-optimised structures in terms of the errors in lattice parameters, T–O bond distances, and T–O–T bond angles (T represents the tetrahedral sites, i.e. the positions occupied by silicon, aluminium, or phosphorus). The resulting errors are then compared to previous calculations using a dispersion-corrected hybrid functional (B3LYP-D2) from the literature [3] and to molecular mechanics calculations employing two different force fields. Furthermore, results for two “outliers” that were removed from the set of reference structures are presented and analysed. Finally, it is assessed how the GGA-type functionals perform in the prediction of the relative stabilities of porous SiO2 polymorphs relative to α-quartz.

2 Materials and methods

2.1 Choice of reference structures

In order to choose a suitable set of all-silica and aluminophosphate zeotypes, only systems fulfilling the following criteria were selected:

  1. 1.

    The material has (nearly) perfect SiO2 or AlPO4 composition, and there are no signs of structural disorder.

  2. 2.

    The experimental structure determination was performed for the calcined material, without structure-directing agents or solvent molecules in the pores.

  3. 3.

    The experimentally determined structure was obtained at room temperature or lower temperatures.

  4. 4.

    The T–O distances in the structures are not too far away from the expected equilibrium distance. For example, the Si–O distances in the all-silica zeolites considered fall between ~1.57 and 1.64 Å. A somewhat larger tolerance was allowed for Al–O and P–O distances in AlPOs. Similarly, systems were ruled out if peculiar deviations of the O–T–O angles from the ideal tetrahedral angle occur (such deviations were observed, for example, in the all-silica zeolite GUS-1 [GON topology], which contains O–T–O angles of ~95 and ~127° [54]).

  5. 5.

    There are no T–O–T angles of 180° in the structure, since these are considered to be the result of disorder around a high-symmetry position. In one prominent example, early determinations of the ferrierite (FER) structure in space group \(Immm\) included one linearly coordinated oxygen atom, for which the angle is fixed to 180° due to the site symmetry [55–57]. It was later shown that a refinement in a space group with lower symmetry results in an angle of ~170° [29] around this oxygen atom [58, 59].

  6. 6.

    Finally, the unit cell should not exceed a certain size to render the DFT calculations feasible. Computations for all-silica faujasite (FAU framework), which has a large unit cell with an edge length of ~25 Å, were performed by using the primitive cell in the DFT calculations. The structure was transformed back to the conventional (face-centred) cell for the comparison with experiment. In all other cases, the conventional cell was used.

Based on these criteria, suitable zeotypes were identified by making use of the ZeoBase database [60] as well the IZA structure database [61]. The final set of reference structures consisted of eight all-silica zeolites and four aluminophosphates. Additionally, two dense polymorphs of SiO2 and AlPO4, α-quartz and α-berlinite, were included. Further calculations were performed for two other zeotypes, all-silica SOD (sodalite) and all-silica AST. However, an analysis of the error in lattice parameters revealed unusually large deviations (in comparison to all other systems), leading to a very large impact of these systems on the overall error. These two zeotypes were thus omitted from the overall error analysis. In order to develop some possible explanations why the chosen DFT approaches perform less satisfactorily in these cases, the results for SOD and AST are discussed separately towards the end of the article.

Table 1 gives an overview of all systems considered in this study, with references to the articles in which the experimental structure data were published. In the large majority of cases, the experimental diffraction measurements used for the structure determination were performed at room temperature, the three exceptions being ITQ-4 (IFR framework, T meas = 30 K), ZSM-22 (TON framework, T meas = 120 K), and AlPO-34 (CHA framework, T meas = 110 K). It is worth noting that the selection of zeotypes is not limited to a certain window size, but contains systems with accessible pores linked by 8-, 10-, or 12-ring windows (and combinations thereof). The structures are visualised in Fig. 1.

Table 1 Overview of systems included in this study
Full size table
Fig. 1
figure 1

Visualisation of all SiO2 and AlPO4 frameworks considered. Structure drawings were prepared with VESTA [79]

Full size image

2.2 Computational details

All DFT calculations were carried out with the CASTEP code (version 7.01), which uses a combination of plane waves and pseudopotentials [62]. “Core” electrons were represented using on-the-fly-generated ultrasoft pseudopotentials, and an energy cut-off of 800 eV was employed, which was found to give very well-converged total energies (the finite basis correction term δE tot/δlog(E cut) was well below 0.01 eV per atom for all systems, indicating very good convergence [63]). For each system, calculations using different numbers of k-points were performed to ensure convergence with respect to the size of the k-point mesh. The k-point meshes that were found to give converged results are included in Table 1.

As mentioned above, the following exchange–correlation functionals were considered:

  • The very popular PBE functional [15], whose deficiencies in predicting lattice parameters have been well documented [16, 17].

  • Two functionals that are based on PBE, but aim at an improved prediction of the structure and properties of solids: WC [19] and PBEsol [20].

  • The PBE functional combined with two different pairwise dispersion correction schemes: while the PBE-D2 approach devised by Grimme uses one fixed set of dispersion coefficients [22], the PBE-TS approach proposed by Tkatchenko and Scheffler scales the dispersion coefficients on the basis of a Hirshfeld partitioning of the DFT electron density [30].

In all cases, the lattice parameters and all atomic coordinates (unless constrained by symmetry) were relaxed in the structure optimisations. Calculations were considered converged when the following criteria were fulfilled simultaneously: Forces on all atoms are below 10−3 eV Å−1, maximal atomic displacement from one optimisation step to the next is less than 5 × 10−4 Å, the energy change from one step to the next is below 10−6 eV, and residual stress is below 5 × 10−3 GPa. CIF files of all optimised structures (generated by CASTEP through the write_cif_structure flag), as well as one representative CASTEP parameter (.param) and structure (.cell) file are supplied as Electronic Supplementary Material.

2.3 Assessment of errors

For any individual structural parameter x, the absolute deviation between DFT calculation and experiment was calculated as:

$$err_{x} = x_{\text{DFT}} - x_{\exp }$$

The following structural parameters were considered in the analysis: the lattice parameters a, b, c, the T–O bond lengths, and the T–O–T angles. Every non-equivalent parameter was counted once in the sums given below, which means that a cubic system contributes only one error value to the calculated overall errors in lattice parameters, whereas an orthorhombic system contributes three. Similarly, bond lengths and angles were not weighted according to the multiplicity. As there are three monoclinic systems among the structures considered, it would in principle also be necessary to include the monoclinic angle in the error analysis. However, it was found that deviations in this angle from the experimental value are always small, regardless of the choice of functional, remaining below 0.5° (~0.5%), which is why the angle was omitted from the further analysis.

In order to evaluate the overall error, both the mean of absolute errors and the mean of signed errors were calculated. The mean of signed errors (MSE) is calculated as [28]:

$${\textit{MSE}} = \frac{1}{{N_{i} }}\mathop \sum \limits_{i = 1}^{{N_{i} }} err_{x,i}$$

The mean of absolute errors (MAE), also referred to as mean absolute deviation, corresponds to [28]:

$${\textit{MAE}} = \frac{1}{{N_{i} }}\mathop \sum \limits_{i = 1}^{{N_{i} }} \left| {err_{x,i} } \right|$$

While the T–O bond lengths and T–O–T angles all fall reasonably close to an average value, the lattice parameters span a range from 5 to 25 Å. This renders it useful to also consider relative deviations, which were evaluated using the root mean square of relative errors (RMSE):

$${\textit{RMSE}} = \sqrt {\frac{1}{{N_{i} }}\mathop \sum \limits_{i = 1}^{{N_{i} }} \left( {100 \cdot \frac{{err_{x,i} }}{{x_{{{ \exp },i}} }}} \right)^{2} }$$

Finally, the largest individual errors (in absolute and, where useful, relative terms) were also evaluated. The errors obtained for the individual systems are compiled in a Microsoft EXCEL file, which is part of the Electronic Supplementary Material.

It is obvious that the accuracy of the experimental data will determine the outcome of the error analysis. As there are no means to evaluate this accuracy on the basis of literature data, it is assumed that the errors in the experimental structure determinations are much smaller than the deviations between DFT and experiment. Nevertheless, two aspects of particular importance should be highlighted in this context:

  1. 1.

    The DFT-optimised structures correspond to the equilibrium structures at 0 K, whereas the experimental structures were determined from datasets measured at room temperature (or, in some instances, lower temperatures, see above). Therefore, a certain difference in the lattice parameters has to be expected due to (positive/negative) thermal expansion of the real system, which is not accounted for in the calculations. The thermal expansion of several of the systems considered in the present work has been studied experimentally, e.g. for α-quartz [64], ITQ-4 (IFR) [68], and AlPO-34 (CHA) [74]. An estimation based on the thermal expansion coefficients determined in those works indicates that the relative difference in lattice parameters between room temperature and 0 K is usually smaller than 0.5%. As we will see below, the deviations between DFT and experiment are typically larger, and it can thus be expected that the neglect of thermal expansion does not affect the overall conclusions.

  2. 2.

    It has been pointed out that the “apparent” Si–O bond lengths that are obtained from X-ray diffraction measurements at room temperature are shorter than the actual bond lengths due to correlated motion of the SiO4 tetrahedra [80]. A correction for these (translational, librational, and screw) effects leads to an increase in the bond length of approximately 0.005 Å with respect to the “apparent” Si–O bond length at room temperature (this correction is even more pronounced for higher temperatures). While this effect has, to our knowledge, not yet been analysed in detail for aluminophosphates, a similar magnitude can be expected on the grounds of the similar bonding situation.

Throughout this study, all comparisons will be made with lattice parameters, bond lengths, and bond angles that were directly taken from experiment, i.e. without applying any corrections. Where appropriate, however, the discussion will take the aforementioned points into account.

3 Results and discussion

3.1 Error analysis I: lattice parameters

Table 2 reports the errors in the lattice parameters obtained for the DFT-optimised structures, giving separate results for SiO2 systems (α-quartz and eight all-silica zeolites) and AlPO4 systems (α-berlinite and four aluminophosphate zeotypes). Furthermore, the mean of signed errors and mean of absolute errors are visualised in Fig. 2. Looking at the overall errors found for the SiO2 systems, it is apparent that, for the PBE, WC, and PBEsol functionals, MSE and MAE are practically equal, i.e. (virtually) all lattice parameters are overestimated. The overall error is reduced from approximately 0.15 Å to slightly below 0.1 Å when moving from PBE to either of the GGA functionals for solids, with PBEsol performing somewhat better than WC. The smallest errors in the range of 0.05 Å are observed for the two dispersion-corrected approaches, PBE-D2 and PBE-TS. Even for these functionals, the values of MSE and MAE remain fairly similar, since the lattice parameters are still overestimated for most systems (an underestimation is found for four/three of 20 individual lattice parameters for PBE-D2/PBE-TS). In accordance with the observations made for the absolute errors, the smallest relative error (RMSE) is found for the PBE-TS functional, closely followed by PBE-D2. Regarding the largest individual errors, the values of LSE and LRE obtained for the best-performing functional, PBE-TS, amount to +0.11 Å and +0.9%, respectively, whereas the corresponding values for PBE are more than twice as large. In other words, the PBE-TS functional predicts all lattice parameters of the nine SiO2 systems with an error of less than 1%, a rather impressive performance.

Table 2 Overall errors in lattice parameters in absolute and relative terms (MSE, MAE, RMSE), as well as largest signed error (LSE) and largest relative error (LRE) found among the individual results
Full size table
Fig. 2
figure 2

MSE and MAE in lattice parameters. For each functional, the bars from left to right represent: MSE (black, filled) and MAE (grey, filled) for SiO2 systems, MSE (black, checkered) and MAE (grey, checkered) for AlPO4 systems

Full size image

For the compounds with aluminophosphate composition, very similar trends in MSE, MAE, and RMSE as for the SiO2 systems are found. However, an increased MAE and fairly large values of LSE/LRE are observed for the dispersion-corrected functionals. A closer look at the individual lattice parameters shows that this is primarily due to the results for AEN, where PBE-D2 and, to a lesser extent, PBE-TS overestimate the length of the a-axis, but underestimate the length of the b-axis. An inspection of the optimised structures reveals rather large deviations from the experimental structure in the dimensions of the eight-ring windows of AEN, which are elliptically distorted. As a similar behaviour is observed for the “outliers”, all-silica SOD and AST, the apparent problems of dispersion-corrected functionals in dealing with systems having distorted windows will be discussed in more detail below.

As a final remark to this section, it has to be reiterated that the differences between the lattice parameters obtained at room temperature and their equilibrium values at 0 K are likely to amount to a few tenths of a per cent. A more accurate error assessment for the different functionals would need to account for this effect; however, the lack of available low-temperature data precludes a quantitative analysis. From a qualitative perspective, it is worth noting that many systems, among them CHA-SiO2, AlPO-34 (CHA), and ITQ-4 (IFR), exhibit negative thermal expansion (NTE) [68, 74, 81, 82]. As the DFT calculations typically deliver larger lattice parameters than experiment, the agreement should improve further if NTE was accounted for.

3.2 Error analysis II: T–O bond lengths and T–O–T angles

The average Si–O, Al–O, and P–O bond lengths, as well as the values of MSE, MAE, and LSE, for all five functionals are compiled in Table 3. In Fig. 3, the distribution of the bond lengths in the experimental structures is compared to the analogous distributions in the structures optimised with the PBE, PBEsol, and PBE-TS functionals (results for WC and PBE-D2 are omitted from the figure because they are very similar to those obtained with PBEsol and PBE-TS, respectively). The overall errors obtained with these three functionals are visualised in Fig. 4.

Table 3 Average Si–O, Al–O, and P–O bond distances and overall errors in absolute terms (MSE, MAE) and largest signed errors (LSE) found among the individual results
Full size table
Fig. 3
figure 3

Distribution of T–O bond lengths among all zeotypes considered. Left Si–O bond lengths in all-silica zeolites. Right Al–O bond lengths (black) and P–O bond lengths (grey) in aluminophosphates. Results for the WC and PBE-D2 functionals are omitted due to their similarity to PBEsol and PBE-TS results, respectively

Full size image
Fig. 4
figure 4

MSE and MAE in T–O bond lengths for PBE, PBEsol, and PBE-TS functionals. For each functional, the bars represent (from left to right): MSE (black, filled) and MAE (grey, filled) for Si–O bonds, MSE (black, checkered) and MAE (grey, checkered) for Al–O bonds, MSE (black, striped) and MAE (grey, striped) for P–O bonds

Full size image

It is worth noting that PBE, PBE-D2, and PBE-TS give practically identical bond lengths. This behaviour is not unexpected, since the dispersion correction should have no significant impact on the representation of interactions between atoms that are directly bonded to each other. It is also apparent that PBEsol and WC predict bond lengths that are systematically shorter than PBE and its dispersion-corrected variants, the typical difference amounting to ~0.007 Å. Furthermore, it is worth noting that the DFT calculations always lead to a much narrower distribution of the distances when compared to experiment. For example, the difference between the shortest and longest Si–O bond in the experimental structures amounts to ~0.07 Å, whereas all DFT-optimised Si–O bond lengths fall within 0.03 Å. This trend is even more pronounced for Al–O and P–O bonds. This is not a surprising finding, as the DFT calculations will always optimise the bonds until they are close to their equilibrium value, because any appreciable deviation from the equilibrium distance will incur an energetic “penalty”. Therefore, unusually short or long bonds observed in the experimental structures will become longer or shorter during the DFT optimisation. Similar observations were made in our previous work on H2O-containing zeolites, where the DFT calculations led to a narrowing of the distribution of Owater–Hwater distances when compared to experiment [41].

A quantitative comparison of the DFT-optimised Si–O distances to experimental values shows that the bond lengths predicted by PBE/PBE-D2/PBE-TS are, on average, approximately 0.015 Å too long, while WC and PBEsol overestimate the bond lengths by roughly 0.01 Å (as discussed previously, the similar magnitude of MSE and MAE corresponds to systematic overestimation). For Al–O bonds, increased values of MSE and MAE are found for all functionals, and there is also a systematic tendency to overestimate the bond length. A different behaviour is observed for the P–O bonds: While the average deviation between DFT and experiment, reflected in the MAE, falls between those observed for Si–O and Al–O bonds, the MSE is much closer to zero, indicating no tendency towards a systematic over- or underestimation. In particular, the WC and PBEsol functionals tend to deliver slightly shorter P–O bonds than experiment.

In line with the aforementioned tendency of DFT to optimise the bond distances towards their equilibrium values, the largest errors are always observed for bonds that are either unusually short (e.g. Si1–O2 in LTA: d exp = 1.568 Å, Si1–O1 in RTE: d exp = 1.577 Å, Al1–O2 in AEN: d exp = 1.650 Å) or unusually long (Al3–O1 bond in AEN: d exp = 1.748 Å) in the experimental structures. As a consequence of the tendency of DFT to give slightly longer Si–O and Al–O bond lengths than experiment, the largest errors always occur in cases where the bond length is overestimated. The opposite is found for P–O bonds: here, the most prominent deviations are found for the P2–O11 and P1–O3 bonds in AEN, which are very long in the experimental structure (d exp ≈ 1.58 Å).

As mentioned above, a correction of the Si–O bond lengths for thermal motion leads to an increase by approximately 0.005 Å with respect to the “apparent” (= experimentally measured) bond length for structures obtained at room temperature [80]. While a detailed assessment of this effect by means of an analysis of the atomic displacement parameters is beyond the scope of the present work, it can be anticipated that the average error of the DFT methods would be reduced further if such a correction was applied.

The average Si–O–Si and Al–O–P bond angles and the values of MSE, MAE, and LSE for the different functionals are compiled in Table 4. The distribution of the bond angles in the experimental structures is compared to the analogous distributions in the DFT-optimised structures obtained with PBE, PBEsol, and PBE-TS in Fig. 5 (again, results for WC and PBE-D2 are omitted), and the overall errors are visualised in Fig. 6.

Table 4 Average Si–O–Si and Al–O–P angles, overall errors in absolute terms (MSE, MAE) and largest signed errors (LSE) found among the individual results
Full size table
Fig. 5
figure 5

Distribution of T–O–T angles among all zeotypes considered. Left Si–O–Si angles in all-silica zeolites. Right Al–O–P angles in aluminophosphates

Full size image
Fig. 6
figure 6

MSE and MAE in T–O–T angles. For each functional, the bars represent (from left to right): MSE (black, filled) and MAE (grey, filled) for Si–O–Si angles, MSE (black, checkered) and MAE (grey, checkered) for Al–O–P angles

Full size image

In a given structure, the equilibrium T–O–T angle depends not only on the electronic configuration of the central oxygen atom, but also on the structural environment (size and shape of building units and windows in the vicinity). Thus, a rather large variation of the angles is observed in the experimental structures, with the Si–O–Si angles ranging from 138 to 167° and the Al–O–P angles ranging from 132 to 173°. The average values amount to 149.9 and 147.5°, respectively. A comparison of the average angles delivered by the different DFT approaches shows that the functionals without dispersion correction typically predict somewhat larger angles than experiment, whereas the dispersion-corrected approaches give smaller angles. This behaviour, which is in line with the observations of a recent study of zeolite MFI with the PBE and PBE-D2 functionals [38], is also reflected in the values of MSE: while the mean of signed errors is positive for PBE, WC, and PBEsol, it has negative values for PBE-D2 and PBE-TS. The overall deviation is fairly similar for all functionals: for Si–O–Si angles, PBEsol performs best, with an MAE of 1.5° compared to MAEs above 2° for all other functionals, but its performance for Al–O–P angles is less impressive. Here, all MAE values fall between roughly 2.5 and 3°. No functional predicts all T–O–T angles within ±5° of the experimental values, with the largest individual errors approaching −10°.

The qualitatively different behaviour of the functionals with and without dispersion correction can be understood from geometric considerations: As stated above, the equilibrium value of a given T–O–T angle will be strongly influenced by the interactions between the building units on both sides, i.e. between atoms that are not bonded to each other, but interact through long-range interactions. In a simple qualitative assessment, one can consider only the T atoms that are bonded to the central oxygen atom, neglecting the remainder of the structural environment. These atoms will be roughly 3 Å apart, a distance for which a significant dispersion interaction will be calculated in pairwise dispersion correction schemes. Because of this attractive dispersion contribution, the equilibrium distance between the T atoms will be optimised towards smaller values if a dispersion correction is applied, consequently leading to a smaller equilibrium T–O–T angle. On the other hand, pure-GGA functionals, which do not account for dispersion interactions, will neglect the attraction between these atoms, thus predicting larger angles. The observed behaviour of both PBE-D2 and PBE-TS indicates that the contribution of dispersion interactions is overestimated by these approaches, a finding that mirrors the observations of some previous benchmarking works [29, 83]. Thus, it appears likely that more evolved dispersion correction schemes incorporating “beyond pairwise” effects may be necessary to accurately capture the intricate balance of short-range and long-range interactions that govern the T–O–T angles [84–86].

3.3 Comparison to other methods I: B3LYP-D2 calculations

The structures of a total of 163 all-silica zeolites were fully optimised with the B3LYP-D2 functional in the work of Coudert, who then used further DFT calculations to predict the elastic properties of a subset of 121 zeolites [3]. Unlike the present plane-wave DFT study, these calculations employed the CRYSTAL code, which uses a local basis set with Gaussian-type basis functions [87]. Since all eight all-silica zeolites considered here were also included in Coudert’s work, we can assess the performance of the hybrid B3LYP functional with a Grimme-type dispersion correction in an analogous manner to the error analysis carried out for the GGA-type functionals. In Table 5, the overall errors of B3LYP-D2 in lattice parameters, Si–O bond lengths, and Si–O–Si angles are given. A direct comparison of the MSE and MAE in lattice parameters to those obtained with PBE, PBEsol, and PBE-TS is made in Fig. 6. While we have to note that α-quartz was not included in the B3LYP-D2 calculations, a direct comparison of the overall errors is nevertheless meaningful because the omission of this system affects the values of MSE and MAE of the GGA-type functionals only marginally.

Table 5 Errors in lattice parameters, Si–O bond lengths, and Si–O–Si angles obtained with DFT calculations using the hybrid B3LYP-D2 functional [3] and with molecular mechanics calculations employing the SLC and ClayFF force fields
Full size table

It is apparent that the B3LYP-D2 functional has a tendency to drastically overestimate the lattice parameters, with both absolute and relative errors that exceed those of the worst-performing GGA-type functional, PBE, by up to a factor of two. While the Si–O bond distances are also overestimated, the typical errors remain in the same range as for PBE and its dispersion-corrected variants. As for the lattice parameters, the errors in the angles are much larger than for all GGA-type functionals, with MSE and MAE exceeding 7.5°. An inspection of the individual results shows that deviations of more than 10° are not uncommon.

While the largest signed error in Si–O–Si angles found among the results exceeds 30°, it has to be pointed out that the most remarkable deviations occur for FER and TON, frameworks where the symmetry of the aristotype, rather than the experimentally observed symmetry, was used in the B3LYP-D2 calculations. Therefore, some angles are 180° for symmetry reasons, being very different from the experimentally measured angles (one example is the Si1–O8–Si2 angle in FER, mentioned above). While these symmetry constraints on some of the T–O–T angles will also affect the predicted equilibrium lattice parameters, a closer look at the individual results reveals that even an exclusion of FER and TON reduces the overall MAE/MSE in lattice parameters of B3LYP-D2 only marginally.

It has to be conceded that the comparability of the B3LYP-D2 results to those of the present work is intrinsically limited due to the use of different types of basis sets, in particular as the quality of the Gaussian-type basis set has a rather strong impact on the resulting structures. In their study of all-silica zeolites, Román-Román and Zicovich-Wilson [34] found considerable changes in the molar volume (up to a few per cent) when moving from a small to a fairly large (triple-zeta) Gaussian-type basis set. Even with the larger basis set, these authors observed a significant overestimation of the molar volume when using the B3LYP functional in conjunction with a dispersion correction. This finding, taken together with the above analysis of the structures from [3], clearly indicates that the observed tendency of B3LYP-D2 to overestimate the lattice dimensions is systematic. Nevertheless, further plane-wave calculations with this functional would be necessary to enable a fully quantitative comparison to the results of the present work. While the computational overhead that comes with the inclusion of a fraction of exact exchange does not appear to be justified for the case of B3LYP, it has to be emphasised that this finding cannot be generalised to other hybrid functionals. For example, the PBE0 functional has been used rather successfully in some computational studies of zeolite structures [27, 35].

3.4 Comparison to other methods II: molecular mechanics calculations

Molecular mechanics calculations are computationally much less expensive than DFT calculations, and various force fields designed for all-silica polymorphs have been devised (see [88] for a comprehensive overview). Naturally, the question arises whether the use of computationally demanding DFT methods is actually needed to arrive at an accurate prediction of the structure of all-silica zeolites, or whether a well-calibrated force field might be able to deliver results of similar (or even better) accuracy. While this aspect cannot be addressed in full within the context of this work, an initial assessment was made, performing additional molecular mechanics calculations employing two widely used force fields: firstly, the force field developed by Sanders, Leslie, and Catlow (SLC) [89], which uses core–shell potentials, and secondly, the more recently developed ClayFF force field [90], which employs a combination of point charges and Lennard–Jones potentials. In a comparative study of 16 different force fields, SLC was found to rank among the best-performing force fields for the structure optimisation of eight-ring zeolites and dense SiO2 polymorphs [88]. ClayFF, which was primarily designed for clays and related layered materials, was included in the present work due to its broad applicability.

The overall errors obtained with these two force fields are included in Table 5, and, for lattice parameters, a comparison to some of the GGA-type functionals is made in Fig. 7. Both force fields predict the lattice parameters with an overall accuracy that is comparable to the PBEsol functional, performing significantly better than PBE, but worse than PBE-TS and PBE-D2. An inspection of the mean of signed errors shows that, in contrast to the DFT approaches, the force fields do not exhibit a systematic tendency to overestimate the lattice parameters: While SLC tends to deliver too small lattice dimensions, the MSE for ClayFF is practically zero, i.e. over- and underpredictions cancel each other out.

Fig. 7
figure 7

Comparison of the performance of three GGA-type functionals to the hybrid B3LYP-D2 functional (results from Coudert’s work [3]) and to two force fields (SLC and ClayFF): MSE (black) and MAE (grey) in lattice parameters for all-silica systems

Full size image

With regard to the Si–O bond lengths, SLC outperforms all DFT approaches, predicting an average Si–O distance that is in excellent agreement with experiment and delivering very small values of MSE and MAE. In this context, it is worth noting that the SLC force field was calibrated on room-temperature structures of SiO2 polymorphs (α-quartz, α-cristobalite, α-tridymite, coesite). While the good agreement for the bond lengths is, therefore, not entirely unexpected, the transferability of this force field is nevertheless impressive. The typical errors of SLC in the Si–O–Si angles are similar to those of the dispersion-corrected variants of PBE, and the predicted average angle is also somewhat smaller than the experimental value. As for PBE-D2 and PBE-TS, this might point to an overestimation of long-range interactions between atoms that are not bonded to each other. In contrast to the good performance of SLC, ClayFF underestimates the Si–O bond lengths significantly, with average deviations in the range of 0.025 Å. The deviations in the Si–O–Si angles are also very large and exceed those of B3LYP-D2. In some instances, a ClayFF optimisation leads to a final structure in which some Si–O–Si angles equal 180°: for example, an optimisation of FER that starts from the experimental structure (space group Pmnn, no linearly coordinated oxygen atoms) delivers a final structure with Immm symmetry and an Si1–O8–Si2 angle of 180°. Overall, the performance of ClayFF is not satisfactory, despite the small MSE and MAE values found for the lattice parameters.

3.5 Discussion of outliers: all-silica SOD and AST

As mentioned previously, rather large deviations between DFT-optimised and experimental structures were found for two all-silica zeolites that were initially included in the set of systems studied, all-silica SOD and AST. As the errors observed for these two systems appear to be systematic and intricately linked to particular structural features, they were omitted from the overall error analysis presented above. Instead, key findings for SOD and AST are discussed separately in this section.

As an initial remark, it is important to note that the symmetry of the experimentally observed structures are reduced with respect to the aristotype of the SOD and AST frameworks, both of which are cubic. In the case of SOD, the experimental structure has rhombohedral symmetry, and the reduction of the symmetry manifests in a distortion of the sodalite (sod) cages [78]: while the two six-ring windows that lie perpendicular to the c-axis retain a regular hexagonal shape, the other six windows are heavily distorted (Fig. 8). Similarly, all 12 six-ring windows bordering the ast cage are equivalent in the cubic aristotype of the AST structure, whereas there are two distinct types of six-ring windows in the experimentally observed, tetragonal structure [77]. Here, in particular the six-rings that lie perpendicular to 〈100〉 are fairly distorted.

Fig. 8
figure 8

Visualisation of the sod cage in SOD and the ast cage in AST, comparing the experimental structures to those optimised with PBEsol and PBE-TS functionals. Selected oxygen–oxygen distances across the six-ring windows are given (in Å)

Full size image

For SOD, the GGA functionals without dispersion correction overestimate the lattice dimensions, with errors in a amounting to roughly 0.2 Å (~1.5%), and errors in c reaching up to 0.4 Å (~6%). The two dispersion-corrected variants give a good prediction of a, but underestimate c by up to 0.2 Å (~3%). While the pure-GGA functionals overestimate the Si–O–Si angles at both oxygen atoms by ~5° or more, PBE-D2 and PBE-TS underestimate them by a slightly smaller amount. The qualitatively different behaviour of the functionals with and without dispersion correction can be understood from a closer look at the sod cage, and especially those six-ring windows that are distorted in the rhombohedral structure. Figure 8 visualises this building unit in the experimentally determined structure and in the DFT-optimised structures obtained with PBEsol and PBE-TS. Relevant atom–atom distances across the window are included in the figure. It is apparent that the window is less distorted in the PBEsol structure than in the experimental structure, whereas the distortion is exaggerated by PBE-TS. These changes in the window distortion also explain the observed trends in the lattice parameters. In line with previously discussed observations, this behaviour can be attributed to the crucial role of dispersion interactions: Since the pure-GGA functionals do not account for dispersion, the weak attractive forces acting between atoms across the window, which stabilise the distortion, are absent, and the optimisation with any of these functionals reduces the distortion. Conversely, PBE-D2 and PBE-TS overestimate the long-range attraction across the window, thus predicting an even stronger distortion than observed experimentally.

In the case of AST, pure-GGA functionals overpredict a by up to 0.2 Å (~2%), and overestimate the Si1–O3–Si2 angle by a significant amount (while the other two angles are predicted rather accurately). The dispersion-corrected approaches deliver too small values of a and too large values of c, and both the Si1–O2–Si1 angle and the Si1–O3–Si2 angle are underestimated. Again, this behaviour can be explained by looking at the distorted six-ring windows of the ast cage, visualised in Fig. 8. As in the case of SOD, PBEsol and other variants without dispersion correction deliver an equilibrium structure that is slightly less distorted than the experimental one, whereas PBE-TS (and PBE-D2) lead to a larger distortion. The same explanation based on the absence/overestimation of dispersion interaction can be invoked here. Since those six-rings for which the distortion is most pronounced lie perpendicular to 〈100〉, the observed changes in unit cell dimensions in structures optimised with dispersion-corrected DFT (shrinkage along a, extension along c, Table 6) are directly correlated with the increased distortion of these six-rings.

Table 6 Comparison of lattice parameters and T–O–T angles obtained in DFT optimisations to experimental values for all-silica SOD and AST
Full size table

3.6 Relative stability of selected SiO2 frameworks

While the present work is primarily targeted at obtaining accurate structures, it is quite useful to also include some energetic considerations. Therefore, the relative stability of SiO2 zeolites was assessed for those systems from the set of reference structures for which experimental data are available. The relative stability is calculated as the difference between the DFT energies (per formula unit SiO2) obtained for a given zeolite and for α-quartz, the thermodynamically stable SiO2 polymorph at ambient conditions. In keeping with previous studies, the difference of the DFT energies ΔE DFT is directly compared to the experimental enthalpy of transition from α-quartz to the zeolite, ΔH exp, thereby ignoring vibrational contributions [34]. For those zeolites for which experimental enthalpy data are available (CHA, FAU, FER, IFR), Table 7 provides values of ΔH exp (from [91]) and ΔE DFT obtained with the PBE-D2 and PBE-TS functionals. Results with pure-GGA functionals are omitted, as these functionals give very small values of ΔE DFT (below 3 kJ mol−1 in all cases). This observation corroborates the previous finding that the inclusion of a dispersion correction is crucial to arrive at a reasonable quantitative prediction of the relative stability [34]. The two dispersion-corrected variants give the correct order of stability (α-quartz–FER–IFR–CHA–FAU); however, the PBE-TS functional overestimates the energy difference by 30–100%. PBE-D2 delivers a much better prediction, with near-quantitative agreement for three of the four systems. This finding might appear somewhat surprising given the previous observation that the Grimme-type D2 dispersion correction typically tends to overestimate the dispersion contribution in periodic systems, because the dispersion coefficients were derived for molecules, not for solids (see e.g. the discussion in [29]). It can thus be hypothesised that the good performance of PBE-D2 is, to some extent, owed to a cancellation of errors: The much too small ΔE DFT obtained by PBE alone is compensated by an overestimated dispersion contribution. While the good agreement between calculations and experiment may be somewhat fortuitous, it would be insightful to study the performance of the PBE-D2 functional in the prediction of relative stabilities of zeotypes in more depth.

Table 7 Comparison of the differences of the DFT energies with respect to α-quartz (ΔE DFT) to experimental enthalpies of transition (ΔH exp)
Full size table

4 Conclusions

The key results of our benchmarking of DFT-GGA for the structure optimisation of neutral-framework zeotypes can be summarised as follows:

  1. 1.

    The well-known overestimation of lattice parameters by PBE is only partially cured in GGA-type functionals for solids, WC and PBEsol, which still exhibit a systematic tendency to overestimate the lattice dimensions. Dispersion-corrected variants of PBE perform significantly better than the uncorrected functionals, with PBE-TS giving particularly good results. It can be concluded that some means to account for dispersion interactions is pivotal to obtain accurate lattice parameters in these framework compounds. Interestingly, PBE-TS outperforms PBEsol and WC even for the dense frameworks, α-quartz and α-berlinite. This observation, together with the previous findings of the benchmarking study of Tran et al. [18], might call for a more systematic study of the performance of dispersion-corrected DFT for dense solids.

  2. 2.

    PBE tends to deliver moderately too long T–O bond lengths, and this behaviour is not affected by the dispersion correction. Since WC and PBEsol give more accurate bond distances, it could be fruitful to combine these functionals with a dispersion correction in future work.

  3. 3.

    Unlike for lattice parameters and bond distances, none of the functionals considered gives completely satisfactory agreement for T–O–T angles. While the pure-GGA functionals tend to give larger angles than observed experimentally, the dispersion-corrected functionals deliver too small angles. This behaviour is likely to result from an overestimation of dispersion interactions between atoms that are relatively close together without being directly bonded to each other. This problem was also identified as the origin of the rather poor performance of dispersion-corrected DFT for two outliers that were removed from the set of reference structures, SOD and AST. In these frameworks, both PBE-D2 and PBE-TS exaggerate the distortion of certain windows, leading to rather large errors in the predicted lattice parameters.

  4. 4.

    A comparison with two widely used force fields shows that PBE-TS, the best-performing functional, delivers more accurate results than SLC, the better of the two force fields. However, the performance is not massively superior, indicating that well-calibrated force fields remain competitive to dispersion-corrected DFT. It thus depends on the task in question whether the computational overhead of DFT is justified or not. With increasing computational power, however, it becomes easier to use DFT routinely for task that were previously addressed with force field methods, e.g. to optimise structure models of zeotypes for which a full refinement from experimental diffraction data is not possible.

  5. 5.

    The relative stability of all-silica zeolites with respect to α-quartz is predicted fairly accurately by the PBE-D2 functional, at least for the limited number of frameworks for which experimental enthalpies are available. While the inclusion of dispersion corrections is absolutely necessary to arrive at realistic energy differences, PBE-TS, the best-performing functional for structural properties, performs considerably worse than PBE-D2. The good performance of the PBE-TS functional in the prediction of structures, but its rather poor performance for energetics, is in line with the observations made in previous studies of molecular crystals: although PBE-TS delivers accurate structures [12], it was found that a combination of optimisations at the PBE-TS level with single-point calculations employing the PBE0 functional and incorporating many-body dispersion (MBD) effects was necessary to arrive at sufficiently accurate lattice energies [11, 86]. The very successful submission of Tkatchenko et al. to the Sixth Blind Test of Organic Crystal Structure Prediction Methods used this approach [92].

Altogether, the accuracy obtained with dispersion-corrected GGA functionals is very encouraging. In particular, the PBE-TS approach, which is more sophisticated and less empirical than PBE-D2, can be recommended as a reasonable default choice for structure optimisations, although additional care should be taken in cases where peculiar structural distortions play a role. Clearly, the results of the present study have also shown that there is space for further improvement, especially in terms of the accurate representation of dispersion interactions. It will thus be useful to assess a wider range of functionals and dispersion correction schemes in due course. However, as it has been highlighted throughout the discussion, the interpretation of the observed accuracy remains somewhat “fuzzy” due to the inherent differences between the DFT-optimised structures, corresponding to 0 K, and the experimental structures, which were obtained at finite temperatures (mostly room temperature). In particular, the average errors in lattice parameters of the best-performing PBE-TS functional approach the (estimated) difference between 0 K and room-temperature lattice dimensions. Consequently, a more exact assessment of the deviations between DFT-optimised and experimental structures would require further experimental work comprising structure refinements on low-temperature datasets.

Finally, it has to be noted that only neutral-framework zeotypes were included in the present study, whereas proton- or cation-exchanged systems are more relevant for many applications (e.g. in catalysis or separation). While it can be expected that the main findings of this work should be transferable to charged-framework zeotypes (aluminosilicates, silicoaluminophosphates, etc.), further testing would be necessary. As many of these materials exhibit fractionally occupied cation sites or mixed occupancies on T sites, the construction of representative structure models that can be used in DFT calculations remains a particular challenge.