Keywords

1 Introduction

Most studies investigating the effect of Transcranial Magnetic Stimulation (TMS) in psychotic disorders have focused on patients with schizophrenia. Schizophrenia is one of the most debilitating mental disorders with a substantial burden of disease [1]. Schizophrenia typically begins in late adolescence or early adulthood and runs a lifelong course characterised by relapses. Symptoms of schizophrenia are usually grouped into positive, negative and cognitive symptoms. Positive symptoms include psychotic symptoms such as hallucinations, delusions, disorganisation of thought and disorganised or catatonic behaviour. Positive symptoms of schizophrenia fluctuate and treatment with antipsychotic medication can often diminish positive symptoms of schizophrenia. However, non-response and non-remission percentages are notably high [2]. Negative symptoms include flattening of affect, alogia, avolition, apathy and social withdrawal. Negative symptoms are very invalidating and about 25% of the patients with schizophrenia suffer from severe and persistent negative symptoms [3]. Treatment options of these negative symptoms are limited and often not effective. Cognitive symptoms include impairments in attention, memory, executive functions and processing speed. These cognitive impairments persist throughout the course of the illness and may co-occur with negative symptoms. Negative and cognitive symptoms can be very debilitating and impair everyday life of patients with schizophrenia.

Due to the limitations of current treatment options for patients with schizophrenia, researchers have explored other treatment modalities, including neuromodulation. Neuromodulation strategies have been studied in various forms, however, in psychotic disorders, treatment with repetitive TMS (rTMS) is the most investigated application. Indeed, in the past two decades, a substantial amount of randomised, controlled trials have investigated the effect of rTMS to treat positive and negative symptoms. Regarding positive symptoms, most rTMS studies have focused on the treatment of auditory hallucinations. Several studies have combined their investigation with pre- and post-treatment assessments of cognitive functioning, in order to determine if treatment with rTMS affects cognition. This chapter reviews the literature with regard to the efficacy and safety of treatment with rTMS of positive and negative symptoms in psychotic disorders. In addition, it reviews the literature available on the effects of rTMS on cognition in psychotic disorders, although this was often not the primary research focus in most studies.

2 rTMS Treatment of Negative Symptoms

2.1 Introduction

Negative symptoms appear to be associated with reduced activation of the prefrontal cortex (PFC), in particular in the dorsolateral prefrontal cortex (DLPFC) [4]. High-frequency rTMS of the prefrontal cortex may treat negative symptoms by increasing local cortical excitability. Over the past decades, several studies have investigated the effect of rTMS on negative symptoms. Some of these studies found a significant improvement of negative symptoms after rTMS, but others failed to find a therapeutic effect. Since 2009, a total of 9 meta-analyses have been performed investigating the effect of rTMS for improving negative symptoms [5,6,7,8,9,10,11,12,13]. The latest and largest meta-analysis, involving 19 studies with a total N = 825, found a moderate treatment effect in favour of rTMS with a mean weighted effect size of 0.64 (0.32–0.96) [11]. Although these results are promising, it remains uncertain if this positive treatment effect is also clinically meaningful, and to which extent the therapeutic effects of rTMS are durable. One study found a positive treatment effect up to 3 months follow-up [14], but most studies did not have a follow-up or only had a short follow-up of up to 2 weeks.

This review aims to clarify the underlying mechanism of action and to investigate which moderators, including rTMS parameters and patient characteristics, increase treatment efficacy.

2.2 Mechanisms of Action of Prefrontal rTMS Treatment of Negative Symptoms

Negative symptoms of schizophrenia have been related to impaired functioning of the prefrontal cortex [15]. Prefrontal high-frequency rTMS may increase brain activity in the stimulated area, as well as in associated areas that are part of the same neural circuit, thereby reducing negative symptoms of schizophrenia. Studies have shown that rTMS can facilitate dopaminergic, GABAergic and glutaminergic neurotransmission [16,17,18], and in so doing may induce plasticity in the brain. In order to investigate the underlying working mechanism of prefrontal rTMS in schizophrenia, several neuroimaging studies have been performed. Two studies combined rTMS treatment with Single Photon Emission Computed Tomography (SPECT) scans, and both studies did not detect any changes in regional cerebral blood flow [19, 20]. One EEG study did find a significant cortical activation with the improvement of negative symptoms [21]. Two fMRI studies did not find statistically significant differences in neuronal activation during a working memory task between sham and active rTMS [22, 23]. One combined treatment and neuroimaging study that found a positive treatment effect [14] also found changes in brain activation between active and sham during an fMRI planning task [24] and a social-emotional evaluation fMRI task [25], accompanied by changes in brain metabolism during a 1H-MRS study [26]. During the planning task, activity in the PFC increased and activity in the posterior brain decreased in the active group as compared to the sham group. During the social-emotional evaluation task, rTMS treatment resulted in reduced activation of striato-fronto-parietal brain areas. Furthermore, a 1H-MRS study conducted among a subgroup of patients found increased glutamate and glutamine (Glx) concentration in the prefrontal cortex after bilateral rTMS in the active group as compared to the sham group. Although results are inconsistent, these neuroimaging studies provide evidence for the underlying rationale of prefrontal rTMS treatment for negative symptoms, namely that it can normalise prefrontal brain activity and metabolism. However, study sizes were small and further neuroimaging research is needed.

2.3 Potential Moderators of Effect

Non-invasive neurostimulation with rTMS can improve negative symptoms, but in order to optimise treatment parameters, it is important to investigate potential moderators of effect. These moderators of effect include rTMS treatment parameters, such as frequency of stimulation or duration of stimulation, as well as patient’s characteristics such as duration of illness.

Studies on rTMS treatment of negative symptoms have all used different rTMS treatment parameters; see Table 7.1 for an overview of randomised controlled trials [14, 20, 27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47]. These studies varied in frequency of stimulation, location of stimulation (frontal, parietal, cerebellar vermis), percentage of motor threshold, duration of stimulation and number of TMS pulses administered. In general, a longer treatment duration of more than 2 weeks and a higher number of TMS pulses administered seem to be more effective [11]. Indeed, there is evidence for impaired cortical excitability, connectivity and plasticity in patients with schizophrenia in all stages of the disease [48]. To improve the efficacy of rTMS, it may be necessary to target neural plasticity, for example by applying a greater number of rTMS stimulations or by increasing treatment duration to enhance treatment response.

Table 7.1 Overview of randomized placebo controlled rTMS studies for the treatment of negative symptoms of schizophrenia
Full size table

Regarding the frequency of stimulation, three studies have investigated low-frequency (1–3 Hz) stimulation of the prefrontal cortex [27, 29, 35] but failed to find an effect. Six studies investigated the effect of 20 Hz prefrontal rTMS [29, 30, 37, 39, 41, 42] but only one study found a significant improvement [41]. Most studies (n = 16) have examined the effect of 10 Hz rTMS, which seems the most promising, as the majority of these studies (n = 10) found a significant improvement of negative symptoms in the rTMS group as compared to the sham group. A recent meta-analysis also found a greater effect size in studies applying 10 Hz prefrontal rTMS as compared to other frequencies [11]. Interesting developments include theta burst stimulation (TBS), which has been investigated by two trials, both of which found a significant treatment effect [39, 41].

The location of stimulation varies, but the majority of trials have investigated rTMS stimulation of the left or bilateral prefrontal cortex, and results were promising (see Table 7.1). Low-frequency stimulation of the right prefrontal cortex [27] and high-frequency rTMS of the parietal cortex [38] did not improve negative symptoms. Interestingly, one study (n = 40) investigating high-frequency rTMS of the cerebellar vermis did find a significant decrease in negative and depressive symptoms in the treatment group [43]. Authors hypothesised that high-frequency rTMS of the cerebellar vermis can, through neural network modulations, increase excitability in the frontal lobe.

Other treatment characteristics include type and dosage of medication. Patients with schizophrenia may use high dosages of medication, including antipsychotics, benzodiazepines and anticonvulsant medication. These medications may interfere with the putative mechanism of action of rTMS, namely increasing excitability and neurotransmitter (including dopamine) release in the prefrontal cortex. The vast majority of patients use antipsychotics to treat positive symptoms, but most antipsychotics have high affinity for dopamine (D2) receptors and thus block dopamine. Indeed, one exploratory study found active rTMS to improve antipsychotic-induced extrapyramidal symptoms (EPS), possibly by increasing dopamine release [49]. Clozapine is an atypical antipsychotic that was shown to be superior in the treatment of refractory schizophrenia. Clozapine, in contrast to most other antipsychotics, shows only weak antagonism to the dopamine D2 receptor. Patients with schizophrenia using clozapine may therefore more readily respond to rTMS treatment. Until now, only one exploratory study has been conducted in a cohort of patients on clozapine participating in the RESIS trial [50]. This study (n = 26) found a significant reduction of the PANSS positive subscale and the PANSS general subscale, but not on the PANSS negative subscale, in patients receiving active rTMS as compared to patients receiving sham rTMS. More research on the effect of type and dosage of medication on rTMS treatment response is warranted.

Besides investigating rTMS parameters as potential moderators of effect, it is also important to explore patient’s characteristics as potential moderators. Exploratory analyses in an earlier meta-analysis found a higher effect in studies that included younger patients with a shorter duration of illness [11]. It may be easier to induce neuroplasticity in younger patients with a shorter duration of illness, and more rTMS studies conducted among patients with a first episode psychosis are required.

2.4 Conclusions

In conclusion, high-frequency prefrontal rTMS has been found to improve negative symptoms in patients with schizophrenia, and this improvement may last up to several months after rTMS treatment. Neuroimaging studies showed rTMS to potentially induce changes in brain activity in prefrontal and connected brain areas, thereby reducing negative symptoms. Although several studies have found a significant improvement of negative symptoms, it remains unclear if the results are clinically significant. Regarding rTMS treatment parameters, a treatment frequency of 10 Hz, a treatment location of the left or bilateral PFC, a longer treatment duration and a larger amount of total TMS pulses administered seem to enhance effectiveness. Regarding patient’s characteristics, younger patients with a shorter duration of illness may respond better to rTMS treatment. Further research is needed to investigate the potential benefits of treatment with clozapine on rTMS treatment response. Future studies should also investigate the underlying neural working mechanism and further establish the most effective combination of rTMS parameters.

3 rTMS Treatment of Positive Symptoms

3.1 Introduction

Neuroimaging studies have shown hyperactivation of language areas of the brain to be involved in hallucinations. More specifically, increased activation of the superior temporal gyrus and of Broca’s area (amongst others) have been consistently observed. Indeed, hyperexcitability of such language-related regions has been hypothesised to be associated with auditory hallucinations. The first attempt to use TMS to reduce the frequency and severity of hallucinations was made by Hoffman and colleagues in 1999 [51]. They investigated the effects of 1 Hz TMS over the temporoparietal cortex in patients with schizophrenia and chronic, medication-resistant auditory-verbal hallucinations (AVHs). After some preliminary and promising results, they conducted a RCT comparing two groups: one group of patients received active TMS, the other received sham TMS. A total of 132 minutes of rTMS was administered over 9 days at 90% of the motor threshold. The hallucination change score improved significantly more in the active as compared to the sham group. In particular, the frequency of hallucinations was reduced by the TMS.

Subsequent research confirmed this effect, although not all studies reported significant improvements due to TMS. Several meta-analyses reported medium effect sizes for active compared to sham TMS [52, 53]. The most recent meta-analysis [10] included 13 studies and showed a statistically significant effect size, albeit of smaller magnitude (standardised mean difference of 0.29). Of note, the authors reported that this result was not stable after sensitivity analysis, and publication bias had a substantial impact on the results. They therefore caution that, even though there may be a therapeutic effect for 1-Hz rTMS on auditory hallucinations in schizophrenia, this needs to be confirmed by large-scale RCTs before this finding can be recommended in clinical practice.

3.2 Mechanism of Action

Most studies that tried to improve hallucinations used 1 Hz stimulation, which was shown to reduce cortical excitability. This was based on neuroimaging studies that showed hyperactivation of superior temporal areas. Indeed, increased levels of excitability or spontaneous fluctuations in auditory (and related) cortex may be associated with hallucinatory activity [54].

Few studies have directly investigated the neural effects of 1 Hz rTMS over the temporoparietal junction (TPJ). Tracy et al. [55] tested the effects of a typical 1 Hz protocol (one session) on auditory brain activation in healthy volunteers. Stimulation with rTMS led to attenuation of the underlying auditory cortex response to the stimulus and a contralateral increase in cortical activity. This supports the underlying rationale for rTMS in hallucinations, i.e. to reduce activation of the auditory-verbal system. However, as the investigators rightly note, the lack of studies investigating immediate (and long-term) neural effects of the rTMS protocol highlights the insufficient knowledge of the effects of rTMS on normal physiology. They also suggest that this, combined with a lack of consensus on clinical trial parameters, may be contributing to the ambivalent data in therapeutic trials.

In a study of patients with schizophrenia and auditory-verbal hallucinations, Bais et al. [56] reported that, compared to sham rTMS, stimulation of the left TPJ resulted in a weaker network contribution of the left supramarginal gyrus to the bilateral fronto-temporal network. In addition, left-sided rTMS resulted in stronger network contributions of the right superior temporal gyrus to the auditory-sensorimotor network, right inferior gyrus to the left fronto-parietal network, and left middle frontal gyrus to the default mode network. The authors interpreted this as follows: the decreased contribution of the left supramarginal gyrus to the bilateral fronto-temporal network may reduce the likelihood of speech intrusions that have been shown to be associated with hallucinations. On the other hand, left rTMS appeared to increase the contribution of functionally connected regions involved in perception, cognitive control and self-referential processing, which may aid coping mechanisms. Although the findings hint to potential neural mechanisms underlying rTMS for hallucinations, the authors emphasise that they need corroboration in larger samples.

3.3 Potential Moderators of Effect

Several potential moderators of effects should be considered. Duration of treatment, number of sessions, location of stimulation, and use of sedatives (such as benzodiazepines) have all been suggested in the literature to possibly be of relevance. It seems logical to suppose that longer duration of treatment and the higher number of treatment sessions (or the total number of TMS pulses) will be as associated with better treatment outcomes. However, there is no strong evidence to support this, which may be due to our lack of studies with the proper comparisons. With regard to location of stimulation, the evidence supports the posterior superior temporal cortex as most effective target [57].

In recent years, three novel potential moderators of TMS treatment effect have been identified. First, TMS may be more effective in young and female participants [58]. Second, the distance between the scalp (where the TMS coil is held) and the cortex may matter [59]. That is, in people with a larger distance between the scalp and the cortex, TMS has less effect. The scalp to cortex distance can be measured using previously acquired MRI scans. Finally, a number of studies have shown that TMS response may be dependent, in part, on genetic variation. For example, variation in the BDNF gene has been associated with TMS effects in patients treated for depression [60, 61].

3.4 Conclusions

Meta-analysis shows a small but significant effect of rTMS on improving auditory-verbal hallucinations in schizophrenia. Delusions did not improve in those studies. No studies have targeted delusions and their underlying neural substrate specifically with rTMS; thus, this remains to be investigated. For hallucinations, it is imperative that larger effect sizes need to be observed in order to warrant clinical relevance. Further exploration of different parameters (e.g. intensity and frequency of stimulation, use of neuronavigation for coil placement, etc.) is needed.

4 Effect of rTMS on Cognition in Patients with a Psychotic Disorder

4.1 Introduction

Cognitive dysfunction is a core symptom of schizophrenia, and these cognitive deficits can be profound and disabling. Several studies investigating the effect of prefrontal rTMS on negative symptoms also investigated the effect of rTMS on cognitive function. For some studies, the primary focus of investigation was the effect of prefrontal rTMS on cognition in patients with schizophrenia. Initially, investigating the effect on cognition was important to rule out any adverse cognitive effects, as can be the case with ECT treatment, which can cause negative cognitive side effects. Fortunately, until now, no adverse cognitive side effects have been reported. Indeed, some studies have found prefrontal rTMS to improve certain domains of cognitive functioning. In the following section, the effect of prefrontal rTMS on different cognitive domains, including executive functioning, attention, working memory, verbal memory, processing speed, motor speed and social cognition, will be discussed.

4.2 Executive Functioning

Several studies have investigated the effect of rTMS on executive functioning [14, 22, 24, 31, 34, 39, 42, 45, 62,63,64,65]. Frequently used neuropsychological tests to assess executive functioning were verbal fluency tests, the Wisconsin Card Sorting Test (WCST) and the Trial Making Test (TMT). Executive functioning was also assessed with the Tower of London test, the Controlled Oral Word Association Test, the Stroop interference task, the Stockings of Cambridge (SOC) and a spatial working memory task of the Cambridge Neuropsychological Test Automated Battery (CANTAB). Interestingly, three out of the five studies that investigated changes in verbal fluency found a significant improvement of verbal fluency immediately posttreatment or at 2 weeks follow-up in the rTMS group as compared to sham [14, 34, 39, 62, 64]. One study found a trend for improvement as measured by the Stroop test (t = 2.1, df = 12, p = 0.06) [31]. There was no significant change as measured by the other neuropsychological tests. However, a recent meta-analysis [66] found rTMS to improve executive functioning at a trend level as compared to sham treatment (p = 0.08).

4.3 Attention

Three randomised controlled trials investigated the effect of rTMS on attention using the Tübinger Aufmerksamkeitsprüfung (TAP), the d2-attention task and the rapid visual information processing (RVP) task of the CANTAB [22, 42, 63]. None of them found any significant effects, nor did a recent meta-analysis [66].

4.4 Working Memory

Several studies assessed the effect of rTMS on working memory in patients with a psychotic disorder. The neuropsychological tests used were the n-back test, the digit span test, the pattern recognition memory (PRM) as measured by the CANTAB, the digit sequencing task of the Brief Assessment of Cognition in Schizophrenia (BACS) and a visuospatial working memory test [22, 23, 39, 42, 62, 64, 67, 68]. Most studies did not find any significant change in working memory performance between the sham and the real rTMS group. However, two studies applying high-frequency (20 Hz) rTMS found significant improvements in working memory. One study found that rTMS significantly improved 3-back accuracy to targets in the n-back test [68] and one study found a significant improvement in visuospatial working memory [39]. In addition, a meta-analysis on the effects of rTMS on cognition in schizophrenia found the effect of active rTMS to be significantly greater than that of sham rTMS in improving working memory [66].

4.5 Verbal Memory

Four RCTs investigated the effect of rTMS on verbal memory, using a parallel form of the Hopkins Verbal Learning Test, the BACS Verbal Memory test or the Rey Auditory Verbal Learning Test [14, 31, 62, 64]. Two studies did not find any significant changes between both groups [14, 62] and one found a significant change in BACS Verbal Memory scores, caused primarily by a decline of performance in the sham group rather than an improvement in the rTMS group [64]. Finally, one study found a significant improvement in the delayed recall of the verbal learning test in the rTMS group as compared to the sham group at 2 weeks follow-up [31]. The meta-analysis, including these four studies, did not find a significant difference of rTMS on verbal memory performance [66].

4.6 Processing Speed

One study used the BACS Symbol Coding to investigate the effect of rTMS on processing speed in patients in an early stage psychosis [64], and found a significant change in scores in the rTMS group at 2 weeks follow-up as compared to the sham group, caused by an improved performance in the rTMS group and a decreased performance in the sham group. Another study, conducted among patients with schizophrenia that used the Digit Symbol Substitution Test to measure processing speed, did not find any significant change [14].

4.7 Motor Speed

Motor speed was assessed in three studies using the Grooved Pegboard Test, the motor screening and reaction time of the CANTAB or the BACS Token Motor Total [31, 42, 64]. No significant changes in motor speed after rTMS were found as compared to sham treatment.

4.8 Social Cognition

Two studies have investigated the effect of rTMS on social cognition [24, 69]. One study found that facial affect recognition improved significantly in patients with schizophrenia after rTMS treatment as compared to sham treatment [69]. Another study investigated the effect of rTMS on brain activation during ambiguous social-emotional evaluation in patients with schizophrenia [24]. This study did not find differences in performance, but fMRI analysis showed that rTMS treatment resulted in reduced activation of striato-fronto-parietal brain areas, while sham treatment resulted in an increased activation as compared to baseline [24]. The authors speculate that rTMS therefore may normalise an increased brain response to ambiguous emotional stimuli. It is important to further investigate the effect of neuromodulation on social cognition, as many patients with schizophrenia are troubled by deficits in social cognition.

4.9 Conclusions

In conclusion, most studies that investigated the effect of rTMS on cognition in patients with a psychotic disorder did not find any significant change in cognition between the rTMS and sham group. Some studies found improvement in executive functioning and working memory in the rTMS group, and there is evidence that rTMS may help ameliorate deficits in social cognition. It is important to note that no adverse cognitive effects occurred. Future studies on prefrontal rTMS should include neuropsychological tests to further clarify the effect of rTMS on cognition.

5 Safety and Side Effects of rTMS Treatment in Patients with a Psychotic Disorder

Common reported side effects were facial muscle twitching during stimulation and transient headache after stimulation. It is important to note that the conducted studies on rTMS treatment in patients with a psychotic disorder did not report the occurrence of seizures or other life-threatening events. In general, the rTMS treatment was well tolerated.

6 Conclusions

In conclusion, in the past decades, several studies have investigated the effect of rTMS on auditory hallucinations, negative and cognitive symptoms in patients with a psychotic disorder, in particular schizophrenia. There is a growing body of evidence that rTMS can alleviate auditory hallucinations and reduce negative symptoms, although it should be noted that several studies failed to find effects. Furthermore, rTMS may improve cognitive functioning. However, the effect size of treatments with rTMS is not as yet clinically satisfactory, and more studies are needed to establish a durable and clinically meaningful improvement. More research is also needed concerning the neural effects of these non-invasive brain stimulation interventions.