Dear Editor,

We would like to thank Zimmer et al. for their letter [1] regarding our article “Effects of exercise training interventions on executive function in older adults: A systematic review and meta‑analysis” [2]. Within their letter, Zimmer et al. [1] express support for the purpose of our meta-analysis by recognizing its potential to advance the field by informing both clinical application and future clinical trials. Zimmer et al. [1] also raise three concerns for discussion relative to the specific methodological approach, the use of a fixed effect or random effects model, and the literature search period covered by our meta-analytic review. We reply to each of their points and hope this discussion will further contribute to the advancement of this research topic.

1 Methodological Approach

Zimmer et al. [1] argue that the results of our review are unreliable because of the inclusion of multiple effect sizes from single studies and the fact that these were treated as independent units of analysis. As they point out, the inclusion of multiple effect sizes from the same studies is very common, but results in non-independence of effects. They also share information about the relatively new approach (i.e., multiple-level meta-analysis) described by Cheung [3] and others [4, 5]. We agree that the new approach (i.e., multi-level meta-analysis) is more appropriate for future meta-analyses, and that meta-analysis using conventional approaches has acknowledged limitations.

2 Fixed Effect or Random Effects Model

The second point by Zimmer et al. [1] is associated with the fixed effect model used in our review and concern that the model overestimates the precision of the mean effect size. Fixed effect and random effects are two popular approaches to meta-analysis, and there is continued debate over which is most appropriate with some consensus that the decision depends on the specific situation [6,7,8,9,10]. Our decision to use a fixed effect model was partially influenced by feedback we received through the journal’s review process. Given that Borenstein et al. [7] also suggest employing a fixed effect model to calculate the pooled effect size, we, therefore, changed the statistical analysis from a random effects model to a fixed effect models.

Zimmer et al. [1] also claimed that a fixed effect model should not be applied when including different populations or treatments. We agree with this point. Indeed, our review specifically targeted older adults (aged ≥ 55 years) and purposely excluded participants with serious cognitive impairments, so that our demographic estimates were based on similar population parameters. Regarding treatment, although we cannot ensure the variability of exercise intervention in our meta-analysis, a fixed effect assumption is more restrictive than a random effects model; therefore, a fixed effect method seems to better reflect the variation or heterogeneity in the true effects estimated by each trial [8].

In a previous article, researchers suggested that these two models employ similar sets of formulas, and therefore, might yield similar estimates for the various parameters [7]. To explore this question with our data, we further analysed the results using both a fixed effect model and a random effects model. The results were very similar. That is, the overall effect sizes were both significantly different and of similar magnitude and variance [0.21 (95% CI 0.17, 0.26, SE = 0.02) for the fixed effect model and 0.25 (95% CI 0.18, 0.34, SE = 0.04) for the random effects model].

3 Literature Search Period Covered

Zimmer et al. [1] further questioned whether it is appropriate to include studies only published after 2003 and whether this approach conformed with PRISMA guidelines. Although we acknowledge that Zimmer et al. made valuable comments relative to this issue, we would like to emphasize that we clearly stated our purpose and explained why we selected 2003 as the first year of our literature search period for our meta-analysis [2]. Specifically, the seminal meta-analysis of Colcombe and Kramer [11] suggested a strong relationship between exercise training and executive function, and it became a milestone for conducting future research (having been cited 4,282 times as of 25 September 2020). However, the definition of executive function has evolved in the current exercise and cognition literature, resulting in the inclusion of more diversified executive function tasks since 2003. Therefore, we focused on studies published after 2003 since the information typically provided thereafter permitted examination of the effects of exercise training on executive functions divided into four domains (as explained in the Introduction to our article [2]).

Regarding the search approach in relation to PRISMA guidelines, Liberati et al. [12] specifically state that authors should report “the start and end dates for the search of each database” and define the process for selecting studies for search and study selection. To meet these criteria, we clearly stated that the start date was 2003 based on the research purpose that we focused on.

4 Conclusion

Notwithstanding these comments, we appreciate the feedback from Zimmer et al. [1] on our current article, their comments on the field of exercise and cognition, and their methodological suggestions for future studies.