Recent meta-analyses suggest that mindfulness offers many benefits for health and psychological well-being (Carsley et al., 2018; McClintock et al., 2019; Querstret et al., 2020; Vonderlin et al., 2020), even when taught online (Spijkerman et al., 2016). Other meta-analyses suggest that mindfulness promotes prosocial behaviours (Berry et al., 2020; Donald et al., 2019). In particular, mindfulness appears to improve attitudes toward outgroups (members of other social groups than their own; Berry et al., 2023a, b; Hunsinger et al., 2014; Kang et al., 2014; Lueke & Gibson, 2015; Parks et al., 2014), and reduce discriminatory behaviours (Lueke & Gibson, 2016) and aggression (Fix & Fix, 2013; Gillions et al., 2019; Heppner et al., 2008). When mindfulness is viewed from a dispositional perspective (i.e., as a personality trait or disposition), it is also associated with lower levels of aggression (Eisenlohr-Moul et al., 2016; Heppner et al., 2008; Shorey et al., 2014). However, it is important to note that not all researchers agree on the prosocial effects of mindfulness; for example, another meta-analysis suggests that the benefits of mindfulness meditation are limited (Kreplin et al., 2018). Furthermore, practicing mindfulness is generally effortful, and not everyone has the motivation or energy to do it consistently.

What if some of the benefits of mindfulness could require little to no effort? Researchers have proposed that although all individuals have an innate capacity for mindfulness, implicit (automatic) cognitive processes (e.g., through priming) can momentarily leverage this innate potential to activate a temporary state of mindfulness (Bergeron et al., 2016; Bergeron & Dandeneau, 2016). In one paradigm, researchers unobtrusively present participants with mindfulness-related words (e.g., present moment, non-judgement) using a scrambled sentence task. Accordingly, even this type of indirect priming of mindfulness-related concepts seems to yield benefits, such as more positive cognitive and affective responses (Bergeron et al., 2016; Bergeron & Dandeneau, 2016). After experiencing a social (Bergeron et al. 2016) or personal stressor (Bergeron & Dandeneau, 2016), participants in the implicit mindfulness condition (vs. a control condition) showed higher levels of positive affect and situational self-esteem, lower levels of negative affect, perceived stress, physiological arousal, cortisol (a stress-related hormone), and attentional bias toward negative information.

In general, semantic priming tasks are thought to work by subtly activating constructs or mental representations and making them more temporarily or situationally accessible to memory, which then affects cognition, affect, and possibly, behaviour (Smeesters et al., 2010). In the case of the mindfulness priming task, then, one explanation is that the prime temporarily activates mental constructs related to mindfulness, which, by their nature, then induces a state of mindfulness. Some authors suggest that this process in a way activates individuals’ innate capacity for mindfulness, thereby making them more open and willing to face unpleasant situations and emotions in a non-defensive and non-judgmental way (Bergeron et al., 2016; Bergeron & Dandeneau, 2016).

To date, the effects of implicitly activating mindfulness were focused on self-directed outcomes—one’s positive and negative mood, self-esteem, perceived stress, and the like. However, what of behaviours directed toward others, such as interpersonal aggression? First, reviews of priming research highlight the importance of the difference between an attentional focus on self versus on other in the context of priming (Smeesters et al., 2010). Second, looking at behavioural components in addition to cognitive and affective outcomes is also important because whereas the effects of priming on cognition is well established, its effects on behaviour have been the subject of heated debate (Bargh, 2014; Meyer, 2014). Several classical behavioural priming studies, for instance, have failed to replicate (Klein et al., 2014; McCarthy et al., 2018, 2021). Although one meta-analysis found a robust overall effect of priming on behaviour, this effect is small and can vary based on what exactly is primed and other factors (Weingarten et al., 2016a). Replication studies looking at behavioural priming are thus still relevant today as they were a decade ago.

Beyond the importance of investigating the direct effect of behaviour priming, it is also of interest to investigate the boundary conditions of those effects by investigating moderators such as individual differences (Maier et al., 2007; Smeesters et al., 2009; Weingarten et al., 2016b). In the case of priming mindfulness, for example, Bergeron et al. (2016) showed that participants with low trait mindfulness benefited the most from the implicit mindfulness activation after having experienced a relatively intense social stressor (public speaking part of the Trier Stress Task), indicating that subtly activating the universal concepts of “focus,” “non-judgement,” and “letting go” temporarily help those who might not routinely have these habits of mind. The study authors suggest that this is because the implicit activation temporarily helped individuals with low trait mindfulness to experience or perceive the social stressor in a more “mindful-like” manner. Following the same logic, one could ask which personality traits or disposition may moderate the effect of the implicit mindfulness activation on other-directed outcomes, such as aggression.

Indeed, some people may be more likely to benefit from a temporarily induced state of mindfulness. Individuals low in trait/implicit aggression for example would likely not improve since they would generally rarely act aggressively (i.e., there would be a floor effect), whereas highly aggressive people may stand to benefit the most. Similarly, people with good self-control or high working memory should excel at self-regulating to avoid acting aggressively (Hofmann et al., 2009), so they would show similar effects as for people low in aggression. More naturally impulsive people though—those low in self-control and/or working memory—might again benefit the most from the mindfulness prime.

While previous studies using the mindfulness priming procedure have focused on the effects of the implicit activation on the self in a context of an ego threat, the present set of replication studies emphasized outcomes directed toward others, namely aggression. Furthermore, previous studies have only looked at the effects of the mindfulness prime after an ego threat or stressor, not alone, which should also be tested. Therefore, in addition to testing a direct effect of the procedure without an ego threat or stressor, the current research also sought to better understand which personality factors moderate the effect—in other words, by demonstrating for whom the effect appears to be the most beneficial. We attempted to answer these questions in a set of three studies.

Study 1

In the first of three studies, we hypothesized that participants in the implicit mindfulness priming condition would show lower levels of aggression toward others than those in the control priming condition (because of the emphasis on the concepts of letting go, non-judgment, and acceptance). We also hypothesized that the mindfulness priming procedure would relate to lower aggression the most for individuals: low in self-control or working memory (because these people are normally more impulsive), low in trait mindfulness (because these people are less likely to be in a state of mindfulness), or high in trait or implicit aggression (because for individuals low in aggression, there might be a floor effect, meaning it would be difficult to reduce aggression any further).

Method

Participants

Our selection criteria were that participants come from either the USA or Canada. They were compensated $1, and the study was conducted in English. We planned to use t-tests for comparing groups and multiple regressions for testing the moderations. Using the means and standard deviations from Bergeron et al. (2016; for self-esteem), we estimated their Cohen’s d effect size = 0.46 (for posterior measures only). Based on the pwr package (Champely, 2020), we estimated that for t-tests, detecting this average effect size, with 80% power and an alpha level of 0.05, requires 76 participants per group (228 in total). We also estimated that, for multiple regressions, to detect a small effect size of f2 = 0.04 (based on the squared semi-partial correlation reported by Bergeron et al., 2016), with a power of 80% and an alpha level of 0.05 would require a sample size of at least 266 participants. To err on the side of caution, assuming a portion of the data collected on the online platform would be unusable (e.g., due to low-quality answers, mid-study dropouts, or other exclusions), we set the target sample size to 300 on the CloudResearch recruitment platform (formerly TurkPrime; Litman et al., 2017).

Six datasets were merged (joined) through an inner join—three Qualtrics surveys and three Inquisit tasks. Duplicates were addressed with the rempsyc::best_duplicate function, which keeps the duplicate with the least amount of missing values, and in case of ties, takes the first occurrence. The resulting pool of participants consisted of 284 participants with unique worker IDs. We excluded participants with duplicate IP addresses (1), that declined to keep their participation after debriefing (1), or with more than 80% of incorrect responses on the crucial mindfulness priming task (18), for a total of 20 exclusions. We thus analyzed the data of 264 participants (gender: 55.70% women, 29.50% men, 0.00% non-binary, 14.77% missing; country: 95.08% USA, 1.52% missing, 1.14% Canada, 2.27% other). Of the 264 participants, 129 were in the mindfulness priming condition, and 135 in the control condition.

Procedure

This study uses a between-subject design, whereby participants were randomly assigned to either the experimental group (mindfulness priming) or the control group (neutral words). In a first block, all participants completed three scales (trait self-control, trait aggression, and trait mindfulness) in a randomized order. They then completed the implicit aggression and working memory tasks, before being randomly assigned to one of the two priming conditions (mindfulness vs. control priming). Finally, participants completed an outcome measure of behavioural aggression.

Priming Mindfulness

To implicitly activate mindfulness, we used the same word scrambling task as Bergeron et al. (2016) and Bergeron and Dandeneau (2016). The task consists of presenting participants with series of words from which they must select specific words to construct a meaningful sentence. The unchosen word is used as a prime to activate the desired construct. For example, “play present outside we moment” becomes “we play outside” where present moment is the prime word. By focusing participants’ attention on the words most meaningful to the sentence, the task unobtrusively presents prime words to which limited attention is brought. The experimental (mindfulness) condition consisted of eight sentences containing mindfulness primes (e.g., letting go, nonjudgmental, awareness), and four sentences containing neutral primes. The control (neutral) condition consisted of the same 12 sentences, but with neutral primes instead (e.g., table, rope, sky).

Measures

Trait Self-Control

We used the Brief Self-Control Scale – Alternative Version (α and ω in the present study = 0.84; 7 items; Tangney et al., 2004). Example item: “I am good at resisting temptation” (1 — Not at all to 5 — Very much).

Trait Aggression

We used the Brief Aggression Questionnaire (α = 0.81; ω = 0.82; 12 items; Buss & Perry, 1992). Example item: “Given enough provocation, I may hit another person” (1 — extremely uncharacteristic of me to 7 — extremely characteristic of me).

Trait Mindfulness

We used the Kentucky Inventory of Mindfulness Skills (α = 0.88; ω = 0.89, 39 items; Baer et al., 2004). Example item: “I notice when my moods begin to change” (1 — Never or very rarely true to 5 — Very often or always true).

Implicit Aggression

We used the Implicit Association Test, aggression version, available on the Millisecond website (e.g., Banse et al., 2015). This task is considered valid and reliable (Banse et al., 2015). The experimental blocks (3, 4, 6, 7) contain 20, 40, 20, and 40 trials respectively, and the practice blocks (1, 2, 5), 20 trials each.

Working Memory

We used the Self-Ordered Pointing Task available on the Millisecond website (e.g., Gillett, 2007). This task is considered valid and reliable (Ross et al., 2007) and contains 12 blocks (number of trials not applicable).

Aggressive Behaviour (Dependent Variable)

To measure (reactive) aggression, we used a modified version of the Competitive Reaction Time Task (CRTT), also known as Taylor’s Aggression Paradigm, available on the Millisecond website (similar to Denson et al., 2010). This task is considered valid and reliable (Chester & Lasko, 2019) and contains 4 blocks with 1, 8, 8, and 8 trials, respectively.

Data Analyses

To ensure optimal normal distribution of the data, we identified and applied optimal normalizing transformations (excluding the Ordered Quantile Normalization transformation) via the bestNormalize package (Peterson, 2021; Peterson & Cavanaugh, 2020). We also specifically used Welch t-tests, as per recommendations (Delacre et al., 2017), using a critical value of p < 0.05 with two-tailed tests.

There were no missing data. bestNormalize transformed the following variables: aggressive behaviour (square root), trait mindfulness (Box Cox), trait aggression (asinh), working memory (Yeo-Johnson), and implicit aggression (square root). Trait self-control required no transformation. After the transformations, the variables were reasonably normally distributed and homoscedastic in each group. We identified 10 univariate outliers in the control group, and 10 in the experimental group, with group-based median absolute deviations greater than three. These observations were winsorized using the group’s three median absolute deviation value (Leys et al., 2013; Thériault et al., 2023). We also standardized all continuous variables.

For the linear models, the group variable was dummy coded as 0 (control/reference group) and 1 (mindfulness priming group). The model included all interaction terms between the condition variable and the other potential moderators (condition × trait mindfulness, condition × trait self-control, condition × trait aggression, condition × working memory, and condition × implicit aggression), as well as the simple effects of those moderators.

We performed all statistical analyses in R version 4.2.2 (R Core Team, 2022) using the following additional packages: effectsize (Ben-Shachar et al., 2020, 2022), psych (internal reliability analyses; Revelle, 2018), dplyr (data manipulation; Wickham et al., 2021), interaction (moderation figure; Long, 2019), as well as report (Makowski et al., 2022) and rempsyc (Thériault, 2023) for convenience functions (checking univariate assumptions, missing items, t-tests, moderations, tables, etc.).

CRTT Quantification Strategy

The CRTT suffers from a plethora of quantification strategies, which makes it difficult to compare and replicate past findings. Elson and colleagues (Elson, 2016; Elson et al., 2014) report 157 different quantification strategies used by researchers to date. Unfortunately, there is currently no consensus as to the right analytical technique to employ. Current recommendations are to justify and preregister the strategy used.

We used the following quantification method in Study 1 (but also Studies 2 and 3): we multiplied (1) the average volume of all 25 trials by (2) the average duration of all 25 trials, and then normalized the product. We refer to this strategy as “the normalized product of the averages”. The optimal normalization transformation was then identified and applied automatically through the “bestNormalize” package in R (Peterson, 2021; excluding the option to consider the Ordered Quantile Normalization transformation).

To us, using the product of the average volume and average duration makes the most intuitive sense as a representation of the combined effects of one’s “aggressive” behaviour. For example, investigating the subcomponent of the CRTT separately can be misleading—setting the volume to 10 for a duration of 1 s is qualitatively different than setting the volume to 5 for 5 s. Methods using the sum of volume intensity and duration, or the average of all 50 trials (intensity and duration) would yield, in our opinion, misleading aggression scores. Using the above numbers as an example, our normalized product of averages score would compare 10 (10 × 1) vs. 25 (5 × 5), whereas the sum method would compare 11 (10 + 1) vs. 10 (5 + 5), and the average of all trials 5.5 vs. 5. Using the product of intensity and duration, as was done by Bartholow et al. (2005) and Arriaga et al. (2011), takes into consideration the combined effect of both volume and duration components of the aggressive behaviour.

Results

The Welch two-sample t-test testing the difference of aggressive behaviour by condition (MControl = −0.06, MMindfulness = 0.06) suggests that the effect is statistically not significant, and very small (difference = −0.12, 95% CI [−0.36, 0.13], t(258.24) = −0.95, p = 0.342; Cohen’s d = −0.12, 95% CI [−0.36, 0.12]).

For the linear model testing the interactions (moderations), using the performance and see packages (Lüdecke et al., 2021a, b), we assessed that the model residuals were reasonably linear, homoscedastic, and normally distributed, and there were no high collinearity or model-based outliers flagged.

Results of the moderation analyses showed that trait self-control significantly moderated the effect of priming mindfulness on aggression (Table 1 and Fig. 1). Simple slope analyses (with −1 and + 1 SD; Aiken & West, 1991; Hayes, 2018) revealed that the mindfulness priming condition predicted higher aggression, but only for people high in self-control and not for those at mean or low self-control (Table 2).

Table 1 Exploring other personality moderators of priming mindfulness (Study 1)
Full size table
Fig. 1
figure 1

Simple slopes of the self-control by condition interaction. Note. Interaction effects between trait self-control (at +1/−1 standard deviation) and the experimental conditions on the composite aggression index. The interaction is significant

Full size image
Table 2 Simple slopes of mindfulness priming condition on aggression at −1, mean, and +1 SD of self-control (Study 1)
Full size table

Discussion

Results from this study show that: (a) the mindfulness priming condition (compared to the control condition) did not significantly influence participants’ level of aggression; and (b), of the five personality variables tested, self-control was the only significant moderator that interacted with the experimental condition to predict aggression. Furthermore, we initially hypothesized that priming mindfulness would lower aggression, and more significantly so for those with low self-control. However, the results differ from our predictions: the simple slopes of the interaction showed that priming mindfulness increased aggression for those with high self-control. Indeed, a closer visual inspection of the interaction (Fig. 1) suggests that participants with low self-control in the mindfulness priming condition reported lower aggression than their counterparts in the control condition; however, the corresponding simple slope was not significant. Rather, unexpectedly, the mindfulness priming condition (compared to the control condition) seems to have caused higher aggression for individuals high in self-control.

That a mindfulness intervention may lead to higher aggression is both counterintuitive and contrary to existing literature (Fix & Fix, 2013; Gillions et al., 2019), although in a series of studies, researchers showed that mindfulness induction can reduce guilt and prosocial reparation (Hafenbrack et al., 2022). That this effect arises in people high in self-control is perhaps even more surprising, given that mindfulness typically has beneficial effects even for high self-control individuals (Bowlin & Baer, 2012). Nonetheless, the literature also suggests that trait self-control can sometimes have ironic effects. That is, whereas trait self-control typically relates to positive, desirable effects, it can sometimes lead to counterintuitive (“ironic”) effects, such as negative, undesirable effects when interacting with certain situational circumstances, such as ego depletion (Imhoff et al., 2014; Lindner et al., 2017, however the concept of ego depletion is contested, see, e.g., Friese et al., 2019).

Perhaps a mindfulness prime, which is different from mindfulness practice, is another such condition with ironic effects for high self-control individuals. For example, it could be that participants with high self-control normally inhibit their aggressive impulses, but that the mindfulness priming condition, by emphasizing words like “letting go”, encouraged them to also let go of their usual internal control. In contrast, for people low in self-control, who normally act more impulsively, priming mindfulness might have made them act indeed more mindfully, which in turn could translate to lower aggression.

All in all, these unexpected findings do not align well with previous theoretical predictions of the self-control literature and warrant further investigation. Thus, we aimed to validate these results in a second, confirmatory, preregistered study relying on a larger sample size.

Study 2

Based on the exploratory results from Study 1, we adapted and narrowed our research question and hypotheses to be as parsimonious as possible and only include the three variables of interest: self-control, the priming conditions, and aggression as a dependent variable. For this second, confirmatory study, we speculated that the mindfulness priming procedure might temporarily encourage individuals high in self-control to “let go” and act more naturally, and thus possibly more aggressively. Consequently, we hypothesized that trait self-control will moderate the mindfulness priming procedure, such that individuals high in self-control in the mindfulness priming condition would show higher aggression (compared to those in the control condition or those with low self-control).

Method

Participants

We planned to recruit and analyze a minimum of 342 participants in total after exclusions. We determined this sample size based on a power analysis conducted with the pwr package in R (Champely, 2020), assuming an sr2 (converted to f2) of 0.03 (based on the sr2 value of the self-control interaction effect obtained in Study 1), 1 numerator degree of freedom, 90% power, and a significance level of 0.05. To err on the side of caution, assuming a bit over one-third of the data collected on the online platform would be unusable (e.g., due to low-quality answers, mid-study dropouts, failed attention checks, or other exclusions), we set the target sample size to 513 on CloudResearch.

Three datasets were merged (joined) through an inner join—two Qualtrics surveys and one Inquisit task. Duplicates were addressed with the rempsyc::best_duplicate function, which keeps the duplicate with the least amount of missing values, and in case of ties, takes the first occurrence. The resulting pool of participants consisted of 377 participants with unique worker IDs. We excluded participants with more than 80% of incorrect responses on the crucial mindfulness priming task (10) or who failed the attention checks (5), for a total of 15 exclusions. We thus analyzed the data of 362 participants (mean age = 43.3, SD = 12.70, range: [21, 81]; gender: 59.10% women, 39.20% men, 1.66% non-binary; country: 100.00% USA; race: 76.52% White, 9.67% Black or African American, 7.18% Asian, 4.42% mixed, 2.21% other). Analyses were conducted with n = 178 participants in the mindfulness priming condition and n = 184 in the control condition.

Procedure

We used the same experimental design (between-subject design) and procedure as in the first study: all participants completed scales of trait self-control, trait aggression, and trait mindfulness (in a randomized order), the implicit aggression task, followed by the priming mindfulness task and behavioural aggression task.

Measures

We used the same scales as in Study 1: the Brief Self-Control Scale – Alternative Version (α and ω = 0.84; Tangney et al., 2004), the Brief Aggression Questionnaire (α = 0.83; ω = 0.84; Buss & Perry, 1992), and the Kentucky Inventory of Mindfulness Skills (α and ω = 0.90; Baer et al., 2004). We also used the same Competitive Reaction Time Task (CRTT) and quantification method as in Study 1 for the measure of aggressive behaviour.

Data Analyses

To ensure optimal normal distribution of the data, we again identified and applied optimal normalizing transformations (excluding the Ordered Quantile Normalization transformation) via the bestNormalize package (Peterson, 2021; Peterson & Cavanaugh, 2020). We used a critical value of p < 0.05 with two-tailed tests. We report, as per recommendations, using the rempsyc package (Thériault, 2023), item-level missing values by scale, as well as participants’ maximum number of missing items by scale (Parent, 2013). Trait mindfulness had 0.01% missing data points (with no participant with more than 2 missing items out of 39). No other data were missing. Visual inspection of the missing data using the visdat package (Tierney, 2017) revealed no specific patterns.

As per best practice (van Ginkel et al., 2020), we imputed the two item-level missing values (before calculating the scales means) using the missForest R package (Stekhoven, 2022; Stekhoven & Bühlmann, 2012). bestNormalize (Peterson, 2021) transformed the following variables: aggressive behaviour (square root), trait mindfulness (log), and trait aggression (asinh). After the transformations, the variables were reasonably normally distributed and homoscedastic in each group. We identified one univariate outlier in the control group, and four in the experimental group, with group-based median absolute deviations greater than three, so we winsorized these observations by group to three median absolute deviations (Leys et al., 2013; Thériault et al., 2023).

Results

For the linear model, using the performance and see packages (Lüdecke et al., 2021a, b), we assessed that the model residuals were reasonably linear, homoscedastic, and normally distributed, and that there were no high collinearity or model-based outliers flagged. We tested a linear regression model with an interaction term between self-control and condition (mindfulness priming versus control), which was the only significant interaction in Study 1. Unlike in Study 1, the interaction was not significant (Table 3), meaning that self-control did not moderate the effect of the mindfulness priming task on aggression.

Table 3 Testing the condition × self-control interaction (Study 2)
Full size table

Furthermore, as described in the preregistration, we also tested an alternative exploratory model that contained trait aggression and trait mindfulness, along with two three-way interactions (the condition × trait aggression × trait self-control interaction, and the condition × trait mindfulness × trait self-control interaction). All assumptions were similarly reasonably respected, but none of the three-way or two-way interactions was significant (Table 4). This means that self-control, trait aggression, and trait mindfulness did not moderate the effects of the mindfulness priming task on aggression, even when controlling for other terms. It also means that self-control did not moderate the interaction between the mindfulness priming task and trait aggression, or the interaction between the mindfulness priming task and trait mindfulness.

Table 4 Exploring other personality moderators of priming mindfulness (Study 2)
Full size table

Discussion

As in Study 1, regardless of level of trait self-control, the overall direct effect of the mindfulness condition on aggressive behaviour was not significant. Furthermore, in contrast to Study 1, trait self-control did not moderate the effect of condition on aggressive behaviour, and the effect sizes were practically 0, meaning there was not even a hint of an effect.

The first conclusion indicates that the mindfulness priming condition clearly does not have a direct effect on people’s levels of aggressive behaviour. As in Study 1, unobtrusively priming participants with “mindfulness-related” words did not influence their outward aggressive behaviours. This outcome somewhat contrasts with previous literature, but it is important to note that previous studies (Bergeron et al., 2016; Bergeron & Dandeneau, 2016) showed effects on self-directed outcomes (e.g., positive and negative mood, self-esteem, perceived stress), and not on behaviours focused on others. In this sense, the current results suggest the mindfulness priming procedure might not be strong enough to counter strong other-directed outcomes such as aggression. Perhaps self-reported measures that emphasize internal self-focused dynamics, as opposed to behavioural measures, would be better suited to capture the effects.

Another reason for the non-replication of the moderator effect could be due in part to slight differences in their design, for example through the potential influence of task order and cognitive depletion. Indeed, to narrow our focus in Study 2, we eliminated extraneous variables such as the implicit measure of aggression (aggression IAT) and working memory (self-ordered pointing task), both of which are cognitively demanding. It is possible that the previous addition of these tasks acted as a sort of cognitive depletion before completing the mindfulness priming task, thereby influencing the behavioural outcome of aggression.

Whereas mindfulness priming may lead high self-control individuals to let go, they might not let go so easily, precisely because they are good at controlling themselves. However, if they are first cognitively depleted, they may be more willing to let go when primed to that effect. Consistent with this idea, the literature suggests that brief mindfulness inductions interact with ego depletion to influence aggression (Yusainy & Lawrence, 2015), and furthermore that ego depletion has a stronger effect in high self-control individuals (Imhoff et al., 2014; Lindner et al., 2017). Thus, perhaps the priming effect emerges on more outward-focused behaviours when high self-control participants are cognitively depleted. We test this idea again in the next study.

Study 3

The goal of this third study was to help us better understand the diverging results from Studies 1 and 2. Study 1 suggested that self-control moderates the effect of the mindfulness priming task on aggression, but Study 2 suggested that this was not the case. One way to reconcile these findings is the possibility that the interaction between the mindfulness priming task and self-control only emerges when people are first cognitively depleted (e.g., through the implicit aggression and working memory tasks). To cover the possibility that the results from Study 1 were due to such methodological differences between Study 1 and Study 2, in this third study, we opted for a more exact replication of Study 1 and re-added the implicit aggression and working memory measures to have as close a replication as possible.

We also added two additional measures to see if the moderation effects between self-control and the mindfulness priming task would also extend to self-reported outcomes. First, we added a measure of mood, because it has been shown to be influenced by the mindfulness priming task in earlier studies (Bergeron et al., 2016; Bergeron & Dandeneau, 2016). Second, we also added a state measure of hostility to investigate the possible effects of the priming condition on more self-reported measure of aggression (as opposed to the other-focused measure, i.e., the CRTT). Hostility has also been shown to be associated with mindfulness (Heppner et al., 2008). We added these additional outcome measures at the very end of the procedure to make sure that this would not influence our testing of the original interaction of interest. This way, it would be possible to test both the original hypothesis on behavioural aggression, mood, and state hostility while maintaining a full replication design of Study 1. Based on the Study 1 results, we hypothesized that trait self-control will moderate the mindfulness priming procedure, such that participants with high self-control in the mindfulness priming condition would show higher levels of aggression, negative affect, and hostility than their counterparts in the control condition. For hostility, we also expected to find effects only for the global score and the two subscales of theoretical interest and most sensitive to experimental manipulations (i.e., “feeling mean”, and “aggravation”).

Method

Participants

We used the same parameters as in Study 2. Six datasets were merged (joined) through an inner join—three Qualtrics surveys and three Inquisit tasks. Duplicates were addressed with the rempsyc::best_duplicate function, which keeps the duplicate with the least amount of missing values, and in case of ties, takes the first occurrence. The resulting pool of participants consisted of 475 participants with unique worker IDs. We excluded participants with duplicate IP addresses (3), with more than 80% of incorrect responses on the crucial mindfulness activation task (33), who failed the attention checks (5), or who missed entire sections of the study (2), for a total of 43 exclusions. We thus analyzed the data of 432 participants (mean age = 43.50, SD = 12.80, range: [19, 85]; gender: 58.10% women, 40.50% men, 1.39% non-binary; country: 99.54% USA, 0.46% other; race: 77.78% White, 11.11% Black or African American, 4.17% Asian, 3.47% mixed, 1.39% American Indian or Alaska Native, 2.08% other). Participants were randomly assigned to the mindfulness activation group (n = 214) or to the control group (n = 218).

Procedure

We used the same experimental design (between-subject design) and procedure as in the first study: all participants completed scales of trait self-control, trait aggression, and trait mindfulness (in a randomized order), followed by the implicit aggression, working memory, priming mindfulness, and behavioural aggression tasks, in this order. Additionally in this study, participants also completed measures of mood and state hostility, in this order, after the behavioural aggression task, as additional dependent variables.

Measures

We used the same scales as in Study 1: the Brief Self-Control Scale – Alternative Version (α = 0.83; ω = 0.84; Tangney et al., 2004), the Brief Aggression Questionnaire (α and ω = 0.85; Buss & Perry, 1992), and the Kentucky Inventory of Mindfulness Skills (α = 0.89; ω = 0.90; Baer et al., 2004). However, we also added the Positive and Negative Affect Schedule – Short Version (positive affect α = 0.82, ω = 0.83, negative affect α and ω = 0.92; 10 items; Kercher, 1992) and the State Hostility Scale (α and ω = 0.98; Anderson et al., 1995), which participants completed after the CRTT and before demographic questions. Originally, we only planned to include the State Hostility Scale’s two subscales of theoretical interest and most sensitive to experimental manipulations (i.e., “feeling mean” and “aggravation”, for a total of 21 items; Anderson & Carnagey, 2009) and prepared the online questionnaire accordingly. However, after discussion, we agreed to use all four subscales (35 items) of the State Hostility Scale and wrote as such in the preregistration. However, due to experimenter error, we forgot to update the online questionnaire, so the short version with only the “feeling mean” and “aggravation” subscales were used. Example item: “I feel like yelling at somebody” (1 — Strongly disagree to 5 — Strongly agree).

Data Analyses

To ensure optimal normal distribution of the data, we again identified and applied optimal normalizing transformations (excluding the Ordered Quantile Normalization transformation) via the bestNormalize package (Peterson, 2021; Peterson & Cavanaugh, 2020). We used a critical value of p < 0.05 with two-tailed tests. State hostility had 0.02% missing data points (with no participant with more than 1 missing items out of 21). No other data were missing. Visual inspection of the missing data using the visdat package (Tierney, 2017) revealed no specific patterns.

As per best practice (van Ginkel et al., 2020), we imputed the two state hostility item-level missing values (before calculating the scales means) using the missForest package (Stekhoven, 2022; Stekhoven & Bühlmann, 2012). bestNormalize (Peterson, 2021) transformed the following variables: aggressive behaviour (square root), trait mindfulness (Yeo-Johnson), trait aggression (square root), state hostility (Box Cox), positive affect (Box Cox), negative affect (square root), implicit aggression (Yeo-Johnson), and working memory (Yeo-Johnson). After the transformations, the variables were reasonably normally distributed and homoscedastic in each group, except for negative affect and state hostility, which were still right skewed. We identified 45 univariate outliers in the control group, and 41 in the mindfulness group, with group-based median absolute deviations greater than three, so we winsorized these observations by group to three median absolute deviations (Leys et al., 2013; Thériault et al., 2023).

Results

For the linear model, using the performance and see packages (Lüdecke, Ben-Shachar, et al., 2021a; Lüdecke, Patil, et al., 2021b), we assessed that the model residuals were reasonably linear, homoscedastic, and normally distributed, and that there were no high collinearity or model-based outliers flagged. However, for the models using negative affect and hostility as dependent variables, the quantile-quantile plots suggested that the model residuals were not completely normally distributed (one of the assumptions of such linear models), even after optimal transformation through the bestNormalize package. Critically, none of the interactions of interest, between self-control and the condition term on aggressive behaviour, affect, or state hostility was significant (Table 5).

Table 5 Testing the condition × self-control interaction (Study 3)
Full size table

Furthermore, as described in the preregistration, we also tested alternative exploratory models that examined the influence of the two-way interactions between condition and personality traits (trait mindfulness, trait self-control, trait aggression, working memory, and implicit aggression) on all outcome measures (aggression, positive affect, negative affect, and state hostility). All assumptions were similarly reasonably respected, but none of the two-way interactions was significant (Table 6).

Table 6 Exploring other personality moderators of priming mindfulness (Study 3)
Full size table

Discussion

The results of Study 3 show that, as in Study 1 and Study 2, the mindfulness priming condition does not have direct effects on either behavioural aggression or self-reported mood and hostility toward others. The results also show, as in Study 2, that self-control, trait aggression, implicit aggression, trait mindfulness, or working memory does not moderate the effect of the mindfulness priming condition on aggression, mood, or hostility. Despite sufficient statistical power (as established by our power analyses based on the sr2 effect size from Study 1), we did not replicate the counterintuitive results shown in Study 1. Taken together, these results suggest that the mindfulness priming condition may be too subtle to influence aggressive behaviour, positive and negative affect, and hostility toward others.

General Discussion

In the current set of studies, we wanted to test whether subtly priming the concept of mindfulness would influence behavioural aggression, negative affect, and hostility, either directly or in interaction with key personality variables shown to be associated with aggression. Results of Study 1 showed that priming mindfulness alone was not sufficient to affect behavioural aggression. However, when interacting with self-control, priming mindfulness did affect aggression: for people with low self-control, priming mindfulness related to lower aggression, whereas for people with high self-control, it related to higher aggression.

However, these results could not be replicated in two follow-up studies. Specifically, we attempted to make Study 2 more parsimonious by only including variables of interest: trait self-control and aggression. This reduced design could not replicate the findings from Study 1. Therefore, we hypothesized that something from the design of Study 1 was necessary to generate the effects observed initially. In Study 3, we thus made a close replication of Study 1, while also adding two additional, more subtle dependent variables that should be more easily influenced than behaviour: feelings of hostility as well as positive and negative affect. Even with this close replication, the findings from Study 1 did not emerge. Together, these results (including effect sizes approximating zero in Studies 2 and 3) suggest that the findings from Study 1 might represent a type I error, or in other words, a false positive due for example to chance.

Based on our results, along with existing literature (Bergeron et al., 2016; Bergeron & Dandeneau, 2016), we suggest that the effectiveness of priming mindfulness procedures may be limited to self-directed outcomes that help one deal with an ego threat, and that it does not influence negative or antisocial other-directed outcomes such as aggression. The distinction between self-focus and other-focus in the context of priming is consistent with previous literature (Smeesters et al., 2010), and these results help delineate the extent and limits of brief interventions relying on scrambled word tasks priming mindfulness.

Our results suggest that unobtrusively priming mindfulness does not influence aggression, hostility, or affect. This is somewhat surprising given that previous research demonstrated an effect on several variables, including affect (Bergeron et al., 2016; Bergeron & Dandeneau, 2016). One important difference, however, is that previous studies testing the effects of this priming mindfulness task did so in the context of an ego threat (such as failing an anagrams task, recalling a very negative personal event, or a public speaking task), either before or after having mindfulness implicitly activated. Other research has also shown the protective nature of mindfulness on self-thoughts and self-affect (Britton et al., 2012; Fogarty et al., 2015; Heppner et al., 2008; Huffziger & Kuehner, 2009; Kuehner et al., 2008). In our current set of studies, instead of testing the effects of mindfulness during or after psychological stress, we investigated the boundary conditions of implicit mindfulness in the context of externally directed aggression, hostility, and negative affect.

One could argue that the CRTT, by administering and receiving loud sound blasts, could have acted as a sort of psychological threat. However, this task also acted as our dependent variable, making it impossible to assess its effect on itself. Still, in the third study, participants additionally completed the PANAS, a popular schedule of positive and negative affect, followed by a hostility questionnaire. If the CRTT had acted as a psychological threat, we would have seen its effects on the two subsequent dependent variables, affect and hostility, though we have not. Whereas previous studies (Bergeron et al., 2016; Bergeron & Dandeneau, 2016) showed that participants reported higher positive affect and lower negative affect following psychological stress, our current studies suggest that the same mindfulness priming task does not make people behave or feel less aggressively toward others. One interpretation is that the temporarily prime-induced state of mindfulness is too subtle to influence variables like behaviour, and in particular, aggression, further highlighting the differences between traditional mindfulness practice and this particular scrambled words task.

Finally, there were also minor demographic differences between the studies. In the original studies (Bergeron et al., 2016; Bergeron & Dandeneau, 2016), participants were recruited from the university campus and participants completed the experiment in French, whereas in the current studies, participants were recruited online, from CloudResearch, and completed the experiment in English. Furthermore, the samples differ on other characteristics, such as age (the university sample being about 20 years younger).

These findings also highlight the importance of open science and preregistered replication studies. In the current era of the replication crisis (e.g., Camerer et al., 2018; Ioannidis, 2005; Open Science Collaboration, 2015), preregistered replication studies seem more necessary than ever (Cesario, 2014; Nosek et al., 2018; Nosek & Lakens, 2014). Without them, it is difficult to establish whether original findings—no matter how convincing—are due to chance alone (or worse, to questionable research practices) or whether they are true effects that can be trusted and built upon. Priming effects, in particular, have been difficult to replicate, and some scientists have called for researchers to begin with replicating their own priming effects (Cesario, 2014; Doyen et al., 2012; Ramscar, 2016; Ramscar et al., 2015).

In this sense, the current set of replication studies from our own priming effects constitute a nice case study. Although we conducted the first study transparently and honestly, we were not able to replicate them in follow-up replication studies. There is always a 5% chance of finding a significant finding even when the effect does not in fact exist. Considering researchers’ degrees of freedom further inflates this number. Yet, many researchers may still underestimate the likelihood of false positives and accordingly develop a misplaced confidence in exploratory findings from single studies. Thus, we believe that this set of studies demonstrates the importance of preregistered replication studies, and particularly so in the context of mindfulness and priming research.

Limitations and Future Directions

Study 1 was more exploratory in nature, and as such, was not preregistered, even though multiple tests were conducted, thus increasing the risks of type-I errors (false positives). In Study 2, the instructions for the mindfulness priming procedure were accidentally randomized with the order of the questions, so although the task objectives were obvious, some participants have seen the explicit instructions at different points within that specific task. However, the high success rate for the task suggests this was not a meaningful limitation. In Study 3, the fact that participants completed the CRTT and Positive and Negative Affect Schedule before the State Hostility Scale might have affected their answers on the hostility measure. However, we believe it unlikely to have affected the effect of the experimental priming condition. All three studies were conducted online, as opposed to in the laboratory making it difficult to assess participant’s attention and concentration. Even though some measures were cognitive or behavioural (such as the tasks to measure implicit aggression, working memory, and aggression), all three studies also relied on self-report measures (trait self-control, trait aggression, trait mindfulness, affect, state hostility), which have known limitations, such as being vulnerable to demand characteristics. Finally, readers should avoid generalizing the conclusions from this study to mindfulness as a whole or to intentional mindfulness practice because these are different from the implicit (non-intentional) mindfulness priming used in this study.

Beyond rigorously replicating past research on priming mindfulness using preregistered open science protocols, future research should continue exploring the boundary conditions of this paradigm. We have established that priming mindfulness does not have strong, direct effects on aggression, and that factors such as trait self-control, trait aggression, trait mindfulness, implicit aggression, and working memory, do not moderate its effect on aggression, hostility, or affect. Future research could test whether other dependent variables and moderators are more sensitive to this paradigm, such as positive or prosocial other-focused behaviours, rather than anti-social behaviour like aggression. In particular, future research should test the hypothesis that the effectiveness of the mindfulness prime specifically depends on self-directed outcomes that help one recover from an ego threat. One experimental design for example could compare two conditions: one with an ego threat and one without, before the mindfulness prime, while having both other-focused outcomes (for which it should not work) and self-focused outcomes (for which significant interactions with e.g., mindfulness should come up).

In conclusion, we report mixed findings regarding the effectiveness of a priming mindfulness procedure in relation to different personality traits relative to aggressive behaviour and hostility. A first study showed that self-control moderates the effect of a mindfulness priming task on behavioural aggression, yet two null-results follow-up studies suggest our initial findings may have been a false positive. These findings suggest that scrambled-word-based priming mindfulness tasks do not influence aggression, either alone or in interaction with other personality variables. Instead, the effectiveness of priming mindfulness procedures may be specific to self-directed outcomes that help one recover from ego threat and does not influence negative or antisocial other-focused outcomes such as aggression. It is still unclear, however, whether priming mindfulness would influence positive or prosocial other-focused behaviours. These results also highlight the importance of open science and preregistered replication studies in mindfulness research.