Keywords

1 Brain Imaging: Possibilities and Limitations

Brain imaging today includes a range of methodologies that can reveal various aspects of the physiological basis of cognition, perception and behaviour. Foremost among them are functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). In this chapter, we mainly focus on fMRI and how it can be meaningfully combined with brain stimulation approaches. Towards the end, we return to EEG, its combination with brain stimulation and fMRI and the unique contributions offered by such multimodal research.

fMRI is a non-invasive imaging method, capable of visualising brain areas that are active during different behavioural or cognitive functions. As such, fMRI provides evidence for local (task-dependent) changes in brain activity, but it is limited in revealing direct causal relationships between these brain activity changes and their behavioural or cognitive consequences. Is the observed change in brain activity functionally relevant for the task? To answer this question, the experimental design must somehow be inverted. Where in functional neuroimaging the cognition or behaviour is the independent variable and brain activity the dependent variable, we wish to turn this around. We should manipulate brain activity, making this the experimental factor, and observe the effects of this manipulation on cognition or behaviour. If the experimentally induced brain activity change has effects on task performance, only then can one conclude that the brain activity involved is functionally relevant. The direction of behavioural effects moreover provides information on the potential specific role of the targeted brain region in the task at hand. To achieve this sort of controlled experimental setup, a method of transient and local brain activity manipulation is required. Such methods exist and are collectively referred to as functional brain interference or brain stimulation techniques.

2 Invasive and Non-invasive Brain Stimulation

Brain stimulation techniques (also referred to as brain perturbation, brain interference or neuromodulation techniques) can be divided into invasive and non-invasive approaches. Some invasive methods, such as cooling and microstimulation, are mainly limited to animal studies, while other invasive (deep brain) stimulations are used in humans but only in patient populations. In contrast, transcranial low-intensity electrical current stimulation (tES) and transcranial magnetic stimulation (TMS) are non-invasive brain stimulation techniques (NIBS), which can be safely used in both human volunteers and patients. NIBS allows for controlled manipulation of brain activity in several ways. TES is thought to modulate cortical excitability during application and—depending on stimulation parameters—can also outlast the stimulation itself. Depending on the parameters and form of tES, it can (1) enhance or decrease cortical excitability or (2) align/amplify local oscillations. TMS, depending on the parameters of the application, can (1) induce transient disruptions of neural activity (‘virtual lesions’), (2) enhance or decrease cortical excitability, (3) transiently stimulate (‘activate’) neural populations or (4) even align/amplify local oscillations. In all cases, by transiently changing activity in the stimulated brain area and revealing a subsequent change in a particular behaviour, NIBS can be regarded as a unique research approach to investigate causal structure-function relationships.

2.1 The Physics and Physiology of Single-Pulse Transcranial Magnetic Stimulation (TMS)

Any TMS device consists of a bank of capacitors capable of producing high discharge currents and an electromagnetic stimulating coil to apply magnetic pulses of up to several Tesla. The intense and rapidly changing currents are discharged into the coil, thereby creating a strong and time-varying magnetic field (pulse). This pulse can reach its peak in a few hundred microseconds and induce an electric field in the neuronal tissue underneath the coil. The strength of the induced electric field depends mainly on the rate of change of the magnetic field. Due to the electrical conductivity of the living tissue, the induced electric field results in electrical (eddy) currents in the cortex, in a parallel but opposite direction to the current in the coil (Lenz’s law). These currents can cause depolarisation and induce action potentials, in the underlying neurons.

Physical parameters of the magnetic field (e.g. rise time and spatial field distribution) determine the temporal-spatial characteristics of the magnetic pulse sent into the brain, but the induced electric field characteristics in neural tissue depend on some additional factors. The shape of the skull, the distance from TMS coil to the gyrating cortical layers, the shape of coil and intensity of stimulation and whether pulses are monophasic or biphasic all influence the final effective strength and extent of stimulation at the cortical level. Moreover, the magnetic field strength decreases exponentially with distance and the cortical surface is convoluted. Magnetic coils have different possible geometric shapes, affecting focality and induced current direction. All these characteristics, of stimulation coils and the underlying neuronal tissue, interact to determine the actualised neuronal depolarisation of mostly superficial levels of the brain (within a few cm of the coil). And that is considering the effects of one magnetic pulse only.

2.2 From Single-Pulse to Repetitive TMS: Stimulation Protocols

TMS pulses can be applied one at a time (single-pulse TMS), in pairs separated by a variable interval (paired-pulse TMS) or in multiples, ranging from triple-pulse or even quintuple-pulse TMS. Importantly, for these application methods, the pulses are usually locked to an external event (e.g. task onset), therefore potentially revealing information about the chronometry of a perceptual or cognitive process (e.g. Schuhmann et al. 2009, 2012; de Graaf et al. 2014). We can, therefore, refer to these approaches as chronometric, or event-related, TMS. By applying chronometric TMS at variable times during task execution, it is possible to investigate not only whether a given brain region is necessary for the tested behaviour but also at what time point (with a temporal resolution of 5–10 ms) the neural activity at the stimulation site is critical for successful task performance (chronometry of functional relevance; see also Walsh and Pascual-Leone 2003).

TMS pulses can also be applied rhythmically, for a longer duration, in either ‘conventional’ or ‘patterned’ protocols of repetitive stimulation (repetitive TMS, rTMS, Rossi et al. 2009). The important feature of both conventional and patterned rTMS protocols is that they can modulate the excitability of the stimulated area for some time after the TMS application itself. The nature of these after-effects, whether they are inhibitory or excitatory, mainly depends on the frequency of stimulation. In conventional rTMS protocols, single TMS pulses are applied in a regular rhythm, with a core distinction between low-frequency rTMS (stimulation frequency of 1 Hz or less) and high-frequency rTMS (stimulation frequency >1 Hz). Patterned rTMS refers to repetitive application of short high-frequency bursts of rTMS, interleaved by short pauses of no stimulation. In theta burst stimulation (TBS), short bursts of 50 Hz rTMS are repeated with a rate in the theta range (5 Hz) as a continuous (cTBS) or intermittent (iTBS) train (Di Lazzaro 2008; Huang et al. 2005). Both 1 Hz rTMS and cTBS have been found to produce lasting inhibitory after-effects, whereas high-frequency rTMS and iTBS can induce lasting facilitatory after-effects on motor corticospinal output in healthy participants. These, however, are group-level effects that may differ between participants (e.g. Maeda et al. 2000) and seem dependent on idiosyncratic brain mechanisms (e.g. Chechlacz et al. 2015). The inter- and even intra-individual variability of responses to rTMS is one of the reasons that combining TMS with brain imaging can be so valuable.

The ability of rTMS to induce longer-lasting excitability changes has opened the door for the clinical applications of TMS in treating various neuropsychiatric disorders, for example, by ‘down- or up-regulating’ pathologically hyper- or hypoactive brain areas (Brighina et al. 2003; Haraldsson et al. 2004; Hoffman 2003; Hoffman and Becker 2005; Martin et al. 2003; Paus and Barrett 2004).

2.3 Clinical Applications of TMS

Over the past 20 years, an increasing number of studies of the potential therapeutic effects of TMS have been published (Lefaucheur et al. 2014). Disorders including addiction (Camprodon et al. 2007; Eichhammer et al. 2003), obsessive compulsive disorder (Martin et al. 2003; Sachdev et al. 2001), pain (Khedr et al. 2005; Lefaucheur et al. 2001), schizophrenia (Chibbaro et al. 2005; Lee et al. 2005) and depression (George et al. 1995; Pascual-Leone et al. 1996) have been studied; however, of all the psychiatric disorders, TMS in major depressive disorder (MDD) has been studied most thoroughly.

To treat depression, repetitive TMS is applied to the dorsolateral prefrontal cortex (DLPFC). Numerous studies stimulated either left DLPFC with high-frequency TMS or right DLPFC with low-frequency TMS, with diverse results (for review see, for example, Schonfeldt-Lecuona et al. 2010). O’Reardon et al. (2007) published a large multicentre trial of daily left prefrontal TMS in medication-free patients with MDD, reporting encouraging results. In contrast, Herwig and colleagues found no difference in responder rates or depression rating scales between real TMS and sham treatment groups in their multicentre trial (Herwig et al. 2007). Early meta-analyses of the antidepressant effect of rTMS (Burt et al. 2002; Gross et al. 2007; Holtzheimer et al. 2001; Kozel and George 2002; Martin et al. 2003; McNamara et al. 2001) also revealed mixed results, with differences between findings perhaps relating to small sample sizes as well as heterogeneous designs.

The validity of TMS for the treatment of depression in clinical practice thus remained unclear for quite some time. While TMS certainly seemed to have beneficial effects with therapeutic potential, the inconsistency of results needed explanation, so that consensus could be reached on which TMS protocols are effective for which types of depression patients (see also Ridding and Rothwell 2007). Nevertheless, in 2008, the first rTMS device (NeuroStar TMS Therapy System) received FDA approval for the treatment of resistant major refractory depression in adults. Since then, FDA approval has been awarded to multiple manufacturers for the same rTMS protocols, always stimulating the left frontal cortex with excitatory protocols, daily for several weeks excluding weekends. Since the standard excitatory rTMS protocol takes relatively long to complete (a single session takes 37 min), it is of interest that patterned rTMS (the iTBS protocol described above) received FDA approval as well in summer 2018. iTBS to treat depression achieves similar effects in only a fraction of the time (6 min).

An extensive consensus review on the therapeutic potential and efficacy of TMS was published by Lefaucheur et al. (2014). They concluded that, at that time, there was ‘definite efficacy’ for the antidepressant effects of high-frequency left DLPFC rTMS in the treatment of depression and ‘probably efficacy’ for low-frequency right DLPFC rTMS.

TMS was also considered ‘definitely effective’ in the treatment of neuropathic pain and ‘probably effective’ in the treatment of motor stroke and schizophrenia (Lefaucheur et al. 2014), sometimes through interesting mechanisms. Indeed, clinical studies employed rTMS to alleviate behavioural or cognitive deficits in patients suffering from brain injury, lesions and stroke (see, for example, Brighina et al. 2003; Koch et al. 2008; Oliveri et al. 1999, 2001). By suppressing the intact hemisphere of stroke patients, the damaged hemisphere is (to an extent) released from the strong interhemispheric inhibition. This allows the damaged hemisphere to express its remaining functionality. TMS studies based on this logic have delivered encouraging results, demonstrating that the counterintuitive strategy of decreasing neural excitability of the healthy hemisphere actually improves deficits following unilateral brain damage to the other hemisphere (Brighina et al. 2003; Cazzoli et al. 2010; Koch et al. 2008; Nyffeler et al. 2009; Oliveri et al. 2000a, b, 2001; Shindo et al. 2006; Song et al. 2009).

3 The Multimodal Approach: Combinations of Brain Stimulation and Brain Imaging

Brain imaging and brain stimulation offer highly complementary methods for studying the healthy and diseased human brain. It is, therefore, sensible to combine these approaches in human fundamental and clinical neuroscience. But NIBS and functional imaging can be combined in different ways. Brain imaging can take place before brain stimulation, to guide and/or individually calibrate the brain stimulation protocols. Brain imaging and brain stimulation can be implemented simultaneously, for instance, to chart brain dynamics using TMS pulses as ‘system probes’ or to reveal the neural bases for TMS-induced changes in cognition or behaviour. In some cases, the latter can be achieved also by brain imaging after brain stimulation, if NIBS protocols are used to induce neuronal changes that last sufficiently long to capture them with functional imaging immediately after. Both the simultaneous combination and the different sequential experimental combinations can be considered ‘multimodal approaches’ (see Fig. 14.1). All are useful for the investigation of functional brain-behaviour relationships, though they have different applications, advantages and limitations.

Fig. 14.1
An illustration of a brain with the I P s and F E F regions marked. E E G flows in an M R I head coil and the f M R I and E E G are mapped and end in cognition.

The multimodal approach. Information from individual EEG and fMRI can be used to inform subsequent non-invasive brain stimulation (NIBS), for instance, using fMRI activations to guide the cortical stimulation target or using individual EEG oscillatory activity to calibrate/tailor TMS or TES stimulation protocols. Alternatively, NIBS can be administered simultaneously with brain imaging, such as EEG and fMRI, or even all together. Lastly, the neural effects of, or responses to, NIBS could be evaluated using brain imaging to better understand the neural basis of NIBS effects on cognition and behaviour

Full size image

3.1 Brain Imaging Before Brain Stimulation

When applying TMS in cognitive studies, the brain areas of interest do not always have a behavioural signature output, as is the case for TMS over the motor cortex or visual cortex. For these brain regions, and associated cognitive research questions, it is not straightforward to determine the precise scalp location where TMS pulses should be administered. Functional imaging before TMS can be used to address this problem by precisely localising a task-related area of cortical activation for subsequent use with a frameless stereotaxic TMS neuronavigation system, thus optimising the exact coil positioning for TMS. In this way, the combination of brain imaging and subsequent brain stimulation permits the assessment of whether, in a given participant, this task-related functional activity (shown using brain imaging) is actually functionally relevant to that individual’s successful task performance (Andoh et al. 2006; Sack et al. 2006; Thiel et al. 2005). There are now several commercially available stereotaxic systems for TMS neuronavigation. Most of them allow for fMRI-TMS co-registration procedures so that events occurring around the head of the participant in real space are registered online and visualised in real time at correct positions relative to the participant’s anatomical reconstruction of the brain. By superimposing the functional data on the anatomical reconstruction of the brain, the TMS coil can be neuronavigated to a specific functional activation area of every participant (see Sack et al. 2009) (Fig. 14.2).

Fig. 14.2
An illustration of two 3D human brains positioned inside a head. The first has an activity cluster in a hotspot at the back. The second has a beam in the same place from a posterior view.

fMRI-Guided TMS Neuronavigation. Panel (a) shows several colour-coded fMRI activity clusters superimposed on a reconstruction of the cortical surface, projected within a transparent mesh of a reconstructed head in Talairach space. Each of these clusters represents an individual fMRI ‘hotspot’, that is, strongest task-related activity, of an individual participant obtained in a separate fMRI measurement. The spatial distribution between these individual fMRI activity clusters accounts for the inter-individual variability in structure-function correspondence. Panel (b) shows a snapshot of the BrainVoyager TMS neuronavigation system used to guide TMS coil positioning based on one of these activity clusters of a given participant. The red beam indicates where the magnetic field of TMS is strongest and is navigated in real time to the here orange colour-coded individual fMRI hotspot of this particular participant. The exact positioning of the TMS coil and thus the target area for the magnetic brain stimulation is therefore individually defined based on the fMRI data obtained in a separate session prior to TMS

Full size image

Using such neuronavigation systems, TMS coil positioning can become highly accurate, targeting anatomical or functional ‘hotspots’ in individual participants with millimetre precision. This is relevant since, despite the limited spatial resolution of the applied magnetic field, spatial TMS coil shifts in the order of millimetres have been shown to sometimes result in a complete loss of behavioural or cognitive impairment effects (Beckers and Homberg 1992; d'Alfonso et al. 2002). Comparing different localisation strategies for TMS-based primary motor cortex mappings in terms of accuracy and efficiency, Sparing et al. (2008) found that fMRI-guided stimulation was most precise (accuracy was concluded to be in the millimetre range). Feredoes et al. (2007) used fMRI to localise TMS sites for disruption of short-term verbal information retention.

Sack et al. (2009) investigated the behavioural impact of right parietal TMS on a number comparison task, when TMS localisation was based on (1) individual fMRI-guided TMS neuronavigation, (2) individual MRI-guided TMS neuronavigation, (3) group functional Talairach coordinates or (4) the 10–20 EEG position P4. They quantified the behavioural effect of each TMS localisation approach, calculated the standardised experimental effect sizes and conducted a statistical power analysis, which revealed that the individual fMRI-guided TMS neuronavigation yielded the strongest behavioural effect size (Sack et al. 2009). This increased effect size of TMS when using (f)MRI-guided coil positioning has also been shown in the context of clinical TMS applications for various psychiatric disorders (Ahdab et al. 2010; De Ridder et al. 2011; Herbsman et al. 2009). Development of cortical targeting approaches is still ongoing, with different approaches making different trade-offs between practical and cost feasibility on the one hand and maximising successful cortical targeting on the other hand. To provide a recent example, Duecker et al. (2014) describe an approach that uses pre-existing functional target locations (i.e. published coordinates/location of an fMRI-based group functional hotspot for a task for interest), back-projected onto individual anatomical data after sophisticated cortex-based alignment of those data to an anatomical template. This approach finds a middle ground, marrying some of the benefits of functional localisation and the consideration of individual anatomy, without requiring fMRI measurements in single participants which may not always be available.

3.2 Brain Imaging After Brain Stimulation

Certain brain stimulation protocols, such as rTMS, TBS or anodal/cathodal tDCS, are capable of modulating neural excitability of a region beyond the stimulation duration. Functional imaging can then be used to investigate these prolonged NIBS after-effects. Imaging the immediate and longer-lasting after-effects of NIBS is paramount for revealing the underlying neurobiological mechanisms that cause the observed behavioural changes and clinical treatment effects of TMS stimulation.

An elegant example of this approach comes from Hubl et al. (2008). Here, the right frontal eye field (FEF) was stimulated outside the MR scanner using continuous theta burst rTMS (TBS). Then fMRI was used to map the TBS-induced effects and assess their temporal persistence across the brain during a saccade task. The results showed a TBS-induced suppression of local BOLD activity that appeared 20–35 min (but not immediately) after stimulation (Hubl et al. 2008). Suppression, albeit weaker, was also evident in more remote regions, including the (pre)supplementary and parietal eye fields. Similarly, Cardenas-Morales et al. (2011) used fMRI for exploring the after-effects of iTBS over the primary motor cortex. A recent example demonstrates how this approach is also valuable to understand more cognitive and emotional brain mechanisms and their interplay. Engelen et al. (2018) presented volunteers with short movie clips of actors displaying either neutral or angry whole-body actions, measuring brain activity with fMRI. The left amygdala did not differentiate these stimuli, unless areas in the action network (inferior parietal lobule, ventral premotor cortex) were first inhibited by continuous TBS. The effects of TBS and emotion were also clear from other action network regions, confirming complex dynamics between emotional and action brain systems uniquely revealed by the combination of TBS and subsequent fMRI.

A similar approach has been used to better understand the clinical efficacy of NIBS. For instance, several studies have used functional imaging to visualise TMS after-effects in prefrontal cortex (PFC), to explore the mechanisms underlying therapeutic applications for depression (Fitzgerald et al. 2007). These studies suggest that prefrontal rTMS in normal and depressed participants has profound effects on both local and remote brain regions implicated in depression, including bilateral frontal, limbic and paralimbic areas (Fitzgerald et al. 2007; Kimbrell et al. 1999, 2002; Pogarell et al. 2006, 2007; Speer et al. 2000, 2009; Teneback et al. 1999). Importantly, these rTMS-induced effects appear to be frequency dependent, with low-frequency rTMS leading to bilateral reduction in frontal activation (Fitzgerald et al. 2007).

3.3 Simultaneous Brain Stimulation and Brain Imaging

While useful, functional imaging after TMS application remains fundamentally limited in elucidating the neuronal effects of TMS. Concurrent TMS and neuroimaging offers a broader range of ‘in vivo’ information regarding the actual and immediate effects of TMS on cortical activation, both local and remote. Simultaneous TMS and imaging can thus be used to online track the TMS effects in the brain or probe intracerebral connectivity (Bestmann et al. 2003b, 2004, 2005; Bohning et al. 1999, 2000b; Ruff et al. 2006; Sack et al. 2007). Therefore, even in the absence of overt behaviour, TMS during fMRI facilitates the imaging of pathways of activity spreading within and between brain networks. The simultaneous approach allows the investigation of local and remote brain responses to TMS, and/or the local and remote brain correlates of TMS-induced changes in cognition/behaviour, at a neurophysiological level. Thus, it can be determined, in vivo, which brain areas—either directly or transsynaptically affected by TMS—passively respond to TMS and/or actively underlie the observed TMS-induced behavioural changes during task execution. However, the simultaneous combination of TMS and functional imaging poses great technical challenges. Therefore, it is routinely used by only few research groups, and the number of simultaneous TMS/fMRI publications is still considerably small (see Reithler et al. 2011 for an overview).

Besides the need for specific hardware (e.g. an MR-compatible TMS system), simultaneous TMS and BOLD fMRI require appropriate temporal synchronisation between MRI acquisition and TMS pulse application. Furthermore, the discharge, and even mere presence, of MR-compatible TMS coils in the bore of the magnet produces artefacts in the echo-planar imaging (EPI) images that need to be resolved before the synchronised combination of functional imaging and brain stimulation becomes feasible. We now first briefly outline relevant technical and implementation considerations and then discuss some instructive examples of simultaneous TMS-fMRI experiments that highlight the unique insights offered by this multimodal approach.

3.3.1 Technical Challenges and Practical Implementation

The use of TMS inside the MR scanner during simultaneous TMS/fMRI studies requires several modifications to TMS hardware, specific TMS/fMRI interleaved experimental designs and the consideration or removal of several artefacts. Most importantly, the standard TMS coils routinely used outside the MR scanner are not appropriate for simultaneous TMS/fMRI studies. Instead, specific MR-compatible non-ferromagnetic TMS coils are required in strengthened casing. To fit in the scanner environment, these coils generally have no handle, and positioning requires adaptive solutions since frameless stereotaxy is challenging. A common solution is to fit the coil with MR markers that allow post hoc reconstruction of coil positioning and triangulation of the cortical target. The MR-compatible TMS coil is connected to the stimulator outside the RF-shielded cabin via a cable running through a wave guide. Therefore, the RF shield of the MR scanner is pierced by the TMS cable, which acts as an antenna transmitting RF noise into the scanner. Special RF noise filters then need to be installed for simultaneous TMS/fMRI studies as an additional hardware component. Without an RF filter, the TMS set-up causes a loss of 20–80% in signal-to-fluctuation-noise ratio of EPI images. An RF filter largely prevents this loss, at the cost of around 7% of functional TMS efficacy (Bungert et al. 2012a).

Despite the installation of an RF filter, the MR image quality is often still decreased in simultaneous TMS and fMRI studies. This is because the mere presence of a TMS coil in the scanner can result in static magnetic field inhomogeneities, which particularly affect EPI scans (commonly used for fMRI). Baudewig et al. (2000) systematically investigated the type and extent of the artefacts induced by the TMS coil during MR measurements. The authors revealed that although the anatomical images were unaffected, there were pronounced signal losses and geometric distortions in EPI acquisitions perpendicular to the plane of the coil. However, these artefacts could be markedly reduced, particularly artefacts remote from the coil, by using an EPI orientation parallel to the coil plane. With such EPI orientation, signal losses and geometric distortions attenuate with increasing distance from the coil and so are restricted to the area very close to the coil. In this scenario, it is unlikely that functional images of the human cortex are strongly affected, given a scalp-cortex distance of >1 cm.

After having addressed the technical challenges discussed above, one can progress to the most important step: applying TMS pulses during actual MR EPI data acquisition. But actually, it must be noted that simultaneous TMS/fMRI is not advisable in the strictest sense. In practice, TMS pulses and MRI acquisitions are interleaved, to avoid the artefacts produced by the TMS-induced currents. This way, fMRI scans can remain artefact-free even though TMS is applied in the MR environment. ‘Simultaneous’ or ‘concurrently combined’ TMS/fMRI thus generally refers to interleaved TMS and fMRI measurements. But for all intents and purposes, this may be considered ‘simultaneous’, since the temporal characteristics and resolution of the BOLD signal render the delay ineffectual. To achieve interleaved TMS-fMRI, the MR sequence must send a trigger signal to the TMS apparatus (for instance, through a computer) with every RF pulse excitation. Timing is critical, since distortions may still occur up to 100 ms after administration of a TMS pulse (Bestmann et al. 2003a; Shastri et al. 1999), although this will differ between labs. These lasting artefacts are purportedly related to residual currents in the TMS coil and to currents induced by the vibrations in the TMS coil following a pulse (Shastri et al. 1999). With better vibration absorption in the TMS coil, the delay between TMS pulse and MR image acquisition may be reduced considerably.

There are various methods for temporally interleaving TMS and MRI for simultaneous experiments. For example, TMS pulses and MR images can be interleaved by insertion of temporal gaps after each volume (Ruff et al. 2006; Sack et al. 2007). Sack et al. (2007) applied bursts of rTMS at ~13.3 Hz, in an acquisition gap of 560 ms between subsequent MR volumes. In this study, a delay of 200 ms from the last TMS pulse to the beginning of the next MR volume acquisition protected the subsequent MR acquisition from pulse-related artefacts. Alternatively, TMS pulses can be separated, not by placing them at between functional volume acquisitions but by interleaving them after each slice within one volume (Bestmann et al. 2004, 2005; Bohning et al. 2000a). This method still requires a sufficient delay between TMS pulses and slice acquisition so that subsequent slices are not perturbed. Finally, single slices might also be deliberately perturbed by the TMS pulse and then be identified and replaced, either by interpolation between pre- and post-pulse acquisitions of the same slice or by including affected slices as covariates in a general linear model analysis. Since the latter approach does involve TMS during EPI, researchers should check with their hardware manufacturers whether this procedure is advised. When employing any of these methods with modified EPI sequences to optimise interleaved TMS/fMRI measurements, it is also recommended to introduce oversampling of the phase-encoding direction of EPI images in order to shift the so-called ghosting artefact outside the volume of interest.

One additional problem for simultaneous (interleaved) TMS/fMRI studies was discussed by Weiskopf et al. (2009), who reported that leakage currents may be generated when switching stimulation intensities. In a phantom measurement, these leakage currents in the TMS coil varied parametrically with the TMS output intensity (its capacitor charge) and induced magnetic field inhomogeneities which led to false-positive fMRI findings. In other words, BOLD signal increased parametrically with TMS intensity in their phantom measurement (Weiskopf et al. 2009). Following this report, a technical solution has been pioneered which introduces a relay in parallel (and diodes in series) with the TMS coil. When the relay is closed, leakage current primarily flows through this relay, rather than the TMS coil. A trigger signal then briefly opens the relay so that a TMS pulse can be applied. However, although these (or similar) solutions are now standard in MR-compatible TMS systems, appropriate test measurements should be run in order to identify any remaining artefacts or false positives due to leakage current.

In sum, even an MRI-compatible TMS coil affects the magnetic field (Bungert et al. 2012b), and TMS pulses perturb MR images within a certain timeframe. Several solutions were proposed to limit or circumvent the latter and development is still ongoing. Such development is not limited to the problem of artefacts alone but expands to new design of MR coils to be used with concurrent TMS (Navarro de Lara et al. 2017) or reliable cortical target localisation in concurrent TMS/fMRI, for example (Hubl et al. 2008). Below, we discuss why the benefits of multimodal imaging justify these efforts.

3.3.2 TMS Affects Networks, Not Just a Local Region

Generally, studies using concurrent TMS-fMRI have shown that TMS affects the BOLD signal in the targeted site and moreover task dependently. This is encouraging, given the widespread assumption that TMS affects excitability/activity in the region directly underneath the coil, and that this activity change reflects behavioural effects of TMS (see Reithler et al. 2011, for an exhaustive overview). However, one of the most important additional insights from combined TMS and functional imaging studies is that locally applied TMS not affects neural activity at the stimulation site but also affects remote and interconnected brain regions (Bestmann et al. 2003b; Blankenburg et al. 2008; Bohning et al. 2000a; Denslow et al. 2005; Ruff et al. 2006; Rushworth et al. 2002; Sack et al. 2007). This includes cortical as well as subcortical brain areas, as revealed by early application to the human motor system (Baudewig et al. 2001; Bestmann et al. 2004; Bohning et al. 1999, 2000a). It seems that application of TMS in essence inserts energy into a system; TMS to an isolated neuronal population will excite not only that population, but a connected brain area will propagate the inserted energy throughout its anatomical (Boorman et al. 2007) and functional (Sack 2006) network. It is precisely the value of TMS-fMRI that this spread of TMS excitation can be tracked throughout the brain.

Bohning et al. (1999) showed that the BOLD signal resulting from TMS correlated to the TMS intensity both in local (targeted) and remote brain areas. Moreover, Bohning et al. (2000a) could show that TMS-induced finger movements resulted in BOLD signals throughout the brain that were similar to BOLD signals resulting from voluntary finger tapping. This constituted an early demonstration of the validity of using TMS-fMRI to probe functional/anatomical networks in the brain. Bestmann et al. (2004) confirmed this notion, stimulating with high-frequency rTMS the left primary sensorimotor cortex (M1/S1) at supra- and subthreshold intensities (no finger movements induced in the latter condition) and measuring the BOLD signals throughout the brain. A network of distinct cortical and subcortical motor system structures was activated in response to the TMS, again involving the same regions activated by voluntary finger movements. Interestingly, this was the case even for subthreshold stimulation, showing that TMS can probe an anatomical network even in the absence of overt behavioural response, although subthreshold stimulation in the absence of induced muscle contractions mainly led to enhanced BOLD responses in supplementary and premotor cortices and not in the local M1/S1 region that was actually stimulated (see Hanakawa et al. 2009 for similar intensity-dependent remote activation changes based on spTMS and Caparelli et al. 2010, for remote effects in the visual system).

This suggests that the local BOLD effects, directly underneath the coil, may constitute a special case: they depend on actually induced muscle contractions, while remote connected motor network regions also involved in voluntary movements are activated by M1/S1 TMS even subthreshold (Bestmann et al. 2004; Denslow et al. 2005). Based on modelling work, Esser et al. (2005) suggest that TMS locally stimulates both excitatory and inhibitory neural populations (ergo the net activation and thus BOLD is weaker here) but remotely results mainly in excitatory responses which are easier to detect. However, the matter is not settled, given the still ill-defined intricacies of TMS effects on local neural circuits and moreover the connection between such effects and the BOLD signal (Logothetis 2008; Logothetis et al. 2010). Still, the anatomical and especially functional specificity of the observed remote network effects argues against a non-specific (water ripple-like) spread of TMS-induced activity. Moreover, the observed networks closely resemble the brain systems involved in natural tasks involving the same regions. For a more elaborate review of these issues, see Reithler et al. (2011).

3.3.3 TMS Network Effects Depend on Brain State

Focal TMS can lead to both local and remote neural effects, within anatomically or functionally connected networks. However, several combined TMS/fMRI studies have also found that these effects are state or task dependent. In other words, the state of the brain at the moment of TMS, as induced by task demands or external sensory stimulation, or even by naturally occurring fluctuations, can influence the local and remote network response to TMS. Bestmann et al. (2008) applied TMS over the left dorsal premotor cortex (PMd) at two intensities (low vs. high) and two motor states (grip vs. no-grip). The authors revealed a significant crossover interaction between motor state and TMS intensity over left PMd, arising in right M1 and right PMd. TMS over left PMd during rest (no-grip) led to an activation decrease in right PMd and M1. Such contralateral decrease following TMS has been observed in most (Bestmann et al. 2004; Kemna and Gembris 2003) but not all simultaneous TMS/fMRI studies over the motor cortex (Bohning et al. 2000a; Hanakawa et al. 2009). Importantly, during (left-handed) grip, left PMd TMS actually induced a contralateral increase in activation, with stronger functional coupling following high-intensity TMS as compared to low intensity. This reversal of effects (activation increases/decreases) is likely caused by differences in the initial brain states, in relation to interregional mutual inhibition/facilitation mechanisms (see also O’Shea et al. (2007)).

Recently, state dependence has been demonstrated in more cognitive contexts. Sack et al. (2007) revealed that TMS over right IPS only resulted in right-hemispheric frontoparietal network effects when participants were engaged in a cognitive (spatial judgement) task that required the proper functioning of the targeted brain region. When a control task (colour judgement) did not rely on the parietal cortex, these network effects of TMS were not found (Sack et al. 2007). State-dependent responses to TMS have been shown in the context of spatial attention as well (Bestmann et al. 2007), also concurrently with fMRI (Heinen et al. 2011). Attention is subserved by a frontoparietal functional network, and disruption of this network by TMS can affect attention performance (Dambeck et al. 2006; Duecker et al. 2013; Hilgetag et al. 2001). Simultaneous fMRI revealed BOLD decreases in this network as a whole, associated with TMS-induced attentional bias (Ricci et al. 2012).

Furthermore, recent TMS/fMRI studies showed directional influences of right parietal activity on regions in other networks, like the ventral attention network and fusiform cortex (Leitão et al. 2015), and even frontal cortices, possibly in relation to post-decisional processes or monitoring in the context of a signal detection task (Leitão et al. 2017). These recent examples demonstrate how simultaneous TMS/fMRI continues to provide valuable insights into the neural mechanisms underlying TMS effects on not only motor and perceptual systems but also higher-order cognition such as attention and even memory (Hawco et al. 2017). Collectively, these findings indicate that TMS-induced neural activity is particularly likely to spread to nodes of a (currently active) functional network and that activity does not necessarily spread to regions that are only anatomically connected to the target site.

3.3.4 TMS Network Effects Are Functionally Relevant

The demonstration of remote neural effects of TMS raises the question of whether (and to what extent) these indirect remote effects are also relevant, and functionally related, to the TMS-induced behavioural changes. In other words, are reported behavioural effects of TMS solely attributable to TMS-induced neural activity changes at that target site, or might these behavioural effects relate to a widely distributed network effect? Frontal eye fields (FEF) are the frontal node of the frontoparietal attention network discussed above. Heinen et al. (2011) stimulated right FEF with TMS, in the MR scanner, while participants viewed both face and motion stimuli. When face stimuli were attended, the TMS pulses induced BOLD increases in the fusiform face area. When motion stimuli were attended, the TMS pulses induced BOLD increases in motion area MT+. Here, the FEF-TMS effects on performance correlated with the BOLD changes in local region FEF but also the BOLD changes in remote region MT+.

Ruff et al. (2006, 2008, 2009) also applied TMS to right FEF simultaneously with fMRI. They revealed remote BOLD effects in two bilateral sets of occipital brain regions within retinotopic visual areas V1–V4. Right FEF-TMS led to BOLD increases for peripheral visual field representations, but BOLD decreases for the central visual field. If these remote BOLD effects of TMS were functionally relevant, then assuming that higher BOLD signal equals higher contrast sensitivity, the authors hypothesised that FEF-TMS should enhance peripheral, relative to central, vision. Interestingly, these behavioural predictions were later confirmed by the authors in a psychophysical study outside the MR scanner. This suggests that, indeed, remote effects of TMS can be functionally relevant.

Does that mean that the local and remote BOLD modulations by TMS, discussed above, reflect the operation of, and TMS-impairment of, a functional network? Sack et al. (2007) applied TMS over right and left parietal cortex during whole-brain BOLD fMRI of spatial cognition performance. The authors found that right, but not left, parietal TMS (1) behaviourally impaired spatial cognition and (2) induced BOLD changes across a right-hemispheric network of frontoparietal regions, including right superior parietal lobule (SPL) and ipsilateral middle frontal gyrus (MFG), in specifically the spatial cognition and not a control task. Only SPL had been stimulated by TMS, so it is accurate to state that parietal TMS impaired spatial cognition performance. But when correlating the TMS-induced behavioural impairment (increases in reaction times) to the TMS-induced changes in BOLD activity, the correlations were in fact equally high for the stimulated SPL and the remote MFG. This strongly suggests that the behavioural deficits are not exclusively caused by neural activity changes at the site of stimulation but rather caused by neural network effects. If so, TMS to the other regions in the network could have equivalent effects on behaviour, which can be systematically tested in follow-up experiments (e.g. de Graaf et al. 2009).

Based on such insights, one might turn this logic around, stimulating cortical areas connected to remote areas precisely because those remote regions are hypothesised to be causally relevant, and not directly targetable by TMS, for instance, because they are subcortical. In a breakthrough demonstration, Walsh and Pascual-Leone (2003) stimulated the lateral parietal cortex, based on evidence that hippocampus may interact intensively with the lateral parietal cortex in a wide cortico-hippocampal network to support associative memory function. However, the causal relevance of cortico-hippocampal interactivity had not directly been demonstrated. Walsh and Pascual-Leone (2003) showed that lateral parietal modulation by TMS increased functional connectivity in this network and moreover led to improvements in associative memory. And in fact, when it comes to the dominant current clinical application of TMS, left frontal TMS to treat depression, at least some of the beneficial effects may arise from the remote modulation of limbic system activity through its connections to the stimulated frontal cortex (Fitzgerald et al. 2007).

4 New Developments

Out of the various interesting recent developments, we would like to briefly discuss two (1) closed-loop neuroscience and (2) TMS-fMRI-EEG multimodal implementations. As fMRI developed to become more widespread, EEG itself experienced a resurgence, particularly when it comes to the study of neuronal oscillations and their role in perception, cognition and behaviour. Although we focused on the combination of TMS-fMRI in this chapter and discussed its value at length, these last two new developments involve brain stimulation applied simultaneously with EEG.

4.1 Closed-Loop Neuroscience

Closed-loop neuroscience is an umbrella term that refers to a range of paradigmatic approaches (Zrenner et al. 2016). In the current context, the notion of closed-loop brain stimulation and brain imaging is most important. Brain stimulation changes the state of the brain, but its effects or its efficacy, have in turn been shown to depend on brain state at the time of stimulation. We have discussed this above for TMS-fMRI combinations, but it holds true for EEG as well. For instance, the power (Romei et al. 2008a, b) and phase (Dugué et al. 2011) of ongoing alpha oscillations predict the cortical response to occipital TMS pulses, as indexed by perception of TMS-induced visual experiences called phosphenes. This role of oscillatory phase seems to hold even when it is experimentally controlled by simultaneously applied transcranial alternating current stimulation (tACS). Schilberg et al. (2018) demonstrated this in the motor cortex. This relevance of oscillatory phase might be exploited, for instance, by controlling it with tACS (e.g. Graaf et al. 2020). Goldsworthy et al. (2016) found that the lasting effects of repetitive TMS (continuous theta burst stimulation) depended on the phase of concurrent tACS at which TMS bursts were administered. But oscillatory phase, or any other observable brain state index, can also be measured and analysed in real time to guide brain stimulation. The idea of closed-loop stimulation is that functionally relevant brain state indices are monitored and analysed in real time, and brain stimulation (i.e. TMS pulses) is applied at state-dependent, experimentally controlled, moments in time. Indeed, Zrenner et al. (2018) recently showed that a real-time EEG-TMS set-up enabled stronger lasting effects of a repetitive TMS protocol on motor excitability, by the repeated administration of bursts of pulses at certain phases of the ongoing sensorimotor mu rhythm.

4.2 Simultaneous TMS-fMRI-EEG

In this chapter, we discussed at length how TMS and fMRI can be combined in different ways and how specifically the simultaneous combination of TMS/fMRI allows a very immediate evaluation of how TMS affects the brain locally and remotely by stimulating a specific brain region and simultaneously monitoring whole-brain network changes in activity. But we also showed how such effects were state and task dependent. This is why the closed-loop approach seems promising.

However, simultaneous TMS-fMRI studies ignore the temporal dynamics and rhythmic oscillations of brain activity reflecting (spontaneous) fluctuations between low and high excitable states during which incoming stimuli are less or more efficiently processed and neural signals are less or more propagated within brain networks. In order to understand and unravel these dynamics in detail, the simultaneous combination of EEG with fMRI and TMS makes an invaluable contribution. The core approach is to use EEG to track ongoing oscillatory activity, indexing brain state at the time of a TMS pulse. fMRI then measures the brain-wide response to the TMS pulse. We pioneered and successfully implemented this approach methodologically and showed it to be feasible and safe for human research already in 2013 (Peters et al. 2013).

In our most recent publication (Peters et al. 2020), we demonstrated for the first time how this simultaneous TMS-fMRI-EEG set-up can be applied to fundamental questions within cognitive neuroscience by revealing how TMS as a system probe evokes (sub-)cortical network responses depending on EEG-indexed brain state. To this end, we assessed how trial-by-trial pre-TMS EEG alpha and low-beta power fluctuations influenced motor network activations induced by subthreshold triple-pulse TMS to the right dorsal premotor cortex. Strong pre-TMS alpha power resulted in decreased TMS-induced BOLD activations throughout the bilateral cortico-subcortical motor system (including striatum and thalamus), suggesting shunted network connectivity. In contrast to alpha activity, strong pre-TMS beta power did not impede TMS signal propagation but instead tended to facilitate signal propagation in motor circuits. Such facilitatory effects corroborate and extend recent insights in the key role of beta activity in coordinating communication in the human motor system (Peters et al. 2020). This study clearly demonstrates how concurrent TMS-EEG-fMRI allows to non-invasively chart the communication mechanisms within larger, dynamically interacting networks and subsystems in the brain. The here described simultaneous TMS-fMRI-EEG approach provides a versatile framework also for future studies on the propagation of TMS-induced activity in functional cortico-(sub)cortical networks, allowing to test a wide range of theories through flexible adaptation to the oscillatory band or TMS protocol of interest and its applicability to different network nodes with varying functional relevance. This ability of concurrent TMS-fMRI-EEG also links to the above-described new development in the field of closed-loop neuroscience as it holds the potential to inform and evaluate future EEG-triggered TMS applications aiming to apply TMS at predefined oscillatory brain states that either will or will not lead to specific remote brain activations by either facilitating or shunting signal propagation from the cortical TMS stimulation site.

5 Conclusions

The combination of brain stimulation with brain imaging offers unique experimental possibilities for understanding the functional architecture of the healthy and diseased human brain. Brain imaging before brain stimulation is useful for the identification (in individual participants or patients) of an exact NIBS target site. Brain imaging after brain stimulation is useful for identifying the spatial pattern and persistency of NIBS-induced neural activity changes that last beyond the stimulation itself (NIBS after-effects). Finally, brain imaging during brain stimulation allows one to stimulate a specific brain region while simultaneously monitoring whole-brain changes in brain activity and behaviour, allowing causal brain-behaviour inferences across the entire brain. Simultaneous, or more precisely interleaved, TMS/fMRI studies appear to converge on the following conclusions: (1) focal TMS applied to a brain region has not only local but rather network effects, (2) these network effects of TMS are state and task dependent and (3) these network effects are functionally relevant.

These results seem to have troubling implications for the interpretation of purely behavioural TMS (without concurrent imaging) studies. After all, without, and sometimes even with, concurrent imaging, we cannot determine whether behavioural effects of TMS are due to its local neural effects and the local and remote neural effects which reflect modulation of a functional network, or whether the behavioural effects are even attributable to the remote neural effects rather than effects in the stimulated region? For the latter, there is currently no conclusive evidence, and as discussed elsewhere (Sack 2010), brain stimulation experiments remain highly valuable also without neuroimaging. But these questions and insights prompt us to move away from modular views of brain function and TMS disruption thereof, forcing us to consider a new conceptualisation that involves functional interactions between various remote, connected brain regions, together constituting networks of integrated, functionally relevant activity. Of course, a very positive consequence of this body of work is that TMS imaging can be used to investigate and reveal exactly these mechanisms, to show how interactions within and between brain networks may support perception and cognition. Simultaneous TMS imaging substantially adds information and insight to purely behavioural TMS experiments, without taking away any of the original relevance of such work. In fact, this enterprise can only be enriched by work employing further complementary techniques in combination with brain stimulation, for instance, MR spectroscopy (Stagg et al. 2009), functional near-infrared spectroscopy (Hada et al. 2006; Kozel et al. 2009; Mochizuki et al. 2006) and diffusion-weighted imaging (DWI) of white matter bundles (Boorman et al. 2007; Kloppel et al. 2008).

To complete this viewpoint, and support the system-level investigations outlined above, investigations at a more fine-grained level will likely be required as well. This is achieved most informatively through invasive animal research (e.g. see Funke and Benali 2010), helping us understand the neurophysiological mechanisms underlying the local and remote effects observed in human research. Work with cats (Allen et al. 2007; Aydin-Abidin et al. 2006; de Labra et al. 2007; Moliadze et al. 2005, 2003; Pasley et al. 2009; Valero-Cabre et al. 2007, 2005), rodents (Aydin-Abidin et al. 2008; Trippe et al. 2009) and monkeys (Ohnishi et al. 2004; Hayashi et al. 2004) has already delivered important contributions in this regard, although not yet into the remote effects of TMS. Also, considering the intrinsic intricacies of neural circuits, a multimodal approach with complementary methods (Logothetis 2008) will likely be required to achieve a cross-level understanding of TMS effects in the brain.

The role and potential of TMS in research and therapeutic settings have, thanks in part to the advances described here, not only been validated but actually increased over the years. With the multimodal research facilities now in place in several labs over the world and the analysis on several levels from animal work to human whole-brain analysis to computational modelling, we are starting to improve our understanding of TMS-induced changes in the brain and behaviour. As such, TMS has begun to provide unique insights into the causal relations and interactions within and between system-level networks in the human brain, all in vivo and non-invasively. With the introduction of simultaneous TMS-fMRI-EEG, a new chapter in the book of multimodal imaging has been opened. The new frontier in non-invasive neuroscience research is to integrate the connectome (brain networks) with the ‘dynome’ (brain dynamics) in order to unravel the mechanisms of communication within larger, dynamically interacting networks and subsystems in the brain and to understand how (spatial) network and (temporal) oscillation mechanisms interact. Ultimately, we remain confident that better understanding of the neural effects of TMS will lead to more informed clinical applications. Further effective and well-controlled therapeutic interventions may thus become possible in the near future.