Keywords

1 Introduction

MRIVIEW was originally designed as a software tool for viewing and manipulating volumetric MRI head data and for using this data as an anatomical reference in MEG studies of brain function (Ranken and George 1993). The initial MRIVIEW capabilities included methods for reading in raw MRI data, segmenting structures in the data, reconciling coordinate systems between multiple imaging modalities, viewing combinations of anatomical and functional information, and building models of structures within the head. Since its initial version, MRIVIEW has been extended in several directions, including an MEG/EEG multi-dipole spatiotemporal modeling procedure (CSST), an MEG/EEG Forward Simulator, a greatly expanded set of visualization capabilities, and additional image processing capabilities.

The Calibrated Start Spatio-Temporal (CSST) multi-dipole inverse procedure is based on the Multi-Start Spatio-Temporal (MSST) inverse procedure (Huang et al. 1998). CSST runs multiple nonlinear simplex procedures from random combinations of MRI-derived cortical starting locations. For each set of starting locations, the simplex procedure minimizes a reduced Chi-square value obtained from a linear fit of the dipole timecourses to the measured MEG or EEG sensor data. Using multiple combinations of starting locations allows the procedure to avoid local minima of the reduced Chi-square error function. A parallel version of this procedure has been implemented that uses MPI to distribute the calculation across a Linux cluster. This provides a linear speedup of the procedure on the number of processors, with very little overhead. Graphical interfaces, which are extensions of the MRIVIEW 3D Model Viewer, are used to set up CSST runs and to view and analyze results.

The Forward Simulator (Ranken et al. 2002) allows a user to generate regions of cortical activity using ellipsoidal constraints on a segmented MRI volume and then assign separate timecourses to each of these regions. The timecourses can be sine waves, combinations of several Gaussians, or arbitrary timecourses from input files. The output is either an MEG or EEG forward calculation with user-specified noise based on a sensor geometry derived from a netMEG file.

We begin by describing the computing environment for MRIVIEW, followed by discussions of the MRIVIEW interfaces for segmentation and visualization of MRI data and for visualization of MEG/EEG analysis results. This is followed by an overview of CSST. The Forward Simulator is then described. Finally, we present a case study that uses both the Forward Simulator and CSST to investigate the feasibility of combining MEG and fMRI data to perform analyses of MEG single-pass data.

2 Computing Environment

MRIVIEW is implemented using the programming language and runtime environment IDL (Interactive Data Language) , made by ExelisVIS (www.exelvis.com). IDL is an interpreted language and can be used as an interactive data analysis and visualization environment, but it is mainly used as a fourth-generation scientific programming language and has similarities to Matlab. It supports arithmetic operations on multidimensional arrays and has a wide range of analysis and visualization routines that typically operate on one- to three-dimensional arrays. IDL supports both procedure- and object-oriented programming. One of its most useful features is optional keyword arguments in procedure and function argument lists. This makes it easy to extend the capabilities of existing IDL programs, while maintaining backward compatibility for codes that are already reliant on these programs. IDL also provides an integrated development environment, for code development.

3 Two-Dimensional Interface

When an MRI head data set is read into MRIVIEW, it is maintained as a three-dimensional array that can be viewed in the three orthogonal view planes, to provide standard radiological views (sagittal, coronal, and axial). MRIVIEW can read a broad range of MRI data formats. It converts these data to an MRIVIEW-standard format that relies on IDL’s Save and Restore routines. The 2D Interface in MRIVIEW is used to view MRI data in the user-selected orientation, either two or eight slices at a time, and allows quick paging through the MRI data volume. The 2D Interface is mostly used with the data segmenting capabilities. MRIVIEW provides semiautomated methods for labeling different structures within the head, such as the entire brain, gray matter, white matter, or the scalp. These can be used to create meshes for boundary element method forward models and labeled volumes for finite element of finite difference method models. Figure 1 shows a view of the 2D Interface, with the several head tissue classes labeled, used for a finite difference MEG forward model. Figure 2 shows a 3D rendering of this labeling on the left. MRIVIEW segmentations can also be used for visualization or location categorization purposes. A segmentation of the major brain compartments is shown in 3D on the right of Fig. 2.

Fig. 1
figure 1

The MRIVIEW 2D Interface shown here after performing a segmentation of MRI head data into six tissue types

Full size image
Fig. 2
figure 2

The six tissue class segmentation is shown in 3D on the left. A segmentation of the major brain compartments is rendered in 3D on the right

Full size image

4 Constrained 3D Interface

In order to localize MEG-derived brain activity on brain anatomy, it is often necessary to identify head fiducials in the MRI head data volume. The constrained 3D Interface was initially developed to address this problem. With it, a user can obtain five head surface views (front, back, top, and sides), by selecting an isosurface value. A data structure links these five views to location information, so that cursoring over any of the five surfaces will select the corresponding orthogonal slice views, also shown in the interface (see Fig. 3). After the user selects fiducials corresponding to those used when obtaining MEG (or EEG) data, a MEG (or EEG)-to-MRI coordinate transformation can be obtained and used both for MEG (or EEG) forward (and thus inverse) calculations and to plot representations of source activity on the MRI-based brain anatomy.

Fig. 3
figure 3

The eight-panel view of the constrained 3D Interface, showing selection of the nasion

Full size image

The other major use of the constrained 3D Interface is with the MEG/EEG Forward Simulator. This will be discussed in the Forward Simulator section.

5 Full 3D Interface

A full 3D interface, called the Model Viewer, was developed to show combinations of MRI-derived anatomy with representation of brain activity and sensor locations in a full 3D viewing environment. The Model Viewer consists of a procedural-based graphical user interface (GUI) that makes calls to an object-oriented viewing engine, which in turn utilizes many of the capabilities in the IDL object graphics library. The Model Viewer provides four model objects that are used to store either volumetric or geometric data. Typically, the MRI data is loaded into Model 1, while source representations and other geometric information is loaded into Models 2 through 4. An isosurface of the MRI data can be obtained and sliced in the three orthogonal directions to provide 3D reference anatomy. The slices and isosurface can each be shaded and colored independently and are all stored in a single viewer model (Fig. 4). The Model Viewer can be used to show combinations of anatomy, sensor locations, and magnetic or electric field values by using the available model containers to independently control and then combine model elements, as shown in Fig. 5.

Fig. 4
figure 4

3D Model Viewer user interface (above) and display window (below) showing orthogonal cutplanes

Full size image
Fig. 5
figure 5

MRI-based head anatomy is combined with MEG sensor geometry and a color contour map representation of the magnetic field at the head surface

Full size image

The 3D Model Viewer has a small set of application programming interface (API) routines that can be used to perform more generalized 3D model viewing, including movie generation. The image in Fig. 6 is one frame from a neuron activation simulation movie, showing the changing electric potentials within the neuron and corresponding magnetic fields surrounding it (Blagoev et al. 2007).

Fig. 6
figure 6

A frame from a neuron electric potential and magnetic field simulation, generated using the API capabilities of the Model Viewer

Full size image

6 MEG/EEG Inverse Modeling

Inverse modeling isintegrated into the Model Viewer interface of MRIVIEW, using CSST. In CSST, a multi-dipole inverse procedure is started M times, using M sets of N randomly chosen locations. M and N are user inputs, with N being the dipole model order. In CSST, the cortical voxel locations obtained with the segmentation procedures in the 2D Interface provide the set of locations from which the random starting points are selected. The segmentation procedures can be used to select different types of starting location sets, such as all the brain voxel locations or locations from a selected area of the brain. Originally, cortical locations were typically used, because they provide good coverage of the brain using less than 40,000 points. Currently, the preferred approach uses a calibrated grid that spans the entire brain, with the grid becoming coarser for deeper regions of the brain, since brain activity in these deeper regions cannot be localized as accurately as more superficial sources, based on Cramer-Rao bounds (Mosher et al. 1993). Figure 7 shows a typical grid of starting locations. For each set of starting locations, the multi-dipole procedure implemented in CSST uses the Nelder-Mead nonlinear simplex procedure (Nelder and Mead 1965) to perform a spatial search. For each step of the simplex search, a singular value decomposition is used to obtain a linear fit to the sensor measurements, using a spherical model for MEG or a three-sphere model for EEG, with unconstrained dipole orientations. A reduced Chi-square value is calculated for this fit. This is the value that is minimized by the simplex procedure. The user selects how many of the best fits to save for a given program run. After processing, the Model Viewer in MRIVIEW can be used to display the CSST results (see Fig. 8).

Fig. 7
figure 7

Grid of brain locations from which CSST randomly selects numerous sets of initial dipole locations

Full size image
Fig. 8
figure 8

CSST results for an MEG visual study, showing the best 60, 6-dipole fits from 20,000 starts

Full size image

For higher-dipole model orders (e.g., five to nine dipoles), multi-start procedures become computationally intensive. For this reason, CSST, written in IDL, has been parallelized using the Message Passing Interface (MPI). The parallel version, MPI_CSST, uses a C language implementation of MPI to distribute the multiple starts across a user-selected number of processors in a Linux cluster, each running an instance of CSST. Each instance of CSST runs with its subset of the original set of starting locations. As each CSST process completes, the output is collated with the output from CSST processes that have already completed, to produce a combined output with the number of best fits requested by the user. The performance of MPI_CSST scales linearly with the number of processors used. The speedup obtained using multiple processors makes the real-time use of MPI_CSST feasible.

6.1 Two-Stage Simplex Search

Many starting dipole configurations are far removed from an optimal solution and end up converging on suboptimal solutions after many iterations of the simplex minimization procedure. For this reason, a two-stage simplex approach has been added to MPI_CSST. Stage one uses a coarse convergence setting in the simplex procedure, saving a user-selected number of best solutions. A fine convergence setting is used in the second stage on these best solutions, to obtain the final results. This two-stage approach reduces the analysis time by 50% or more, compared to the single-stage approach.

6.2 MUSIC-Seeded CSST

For difficult MEG/EEG inverse analyses (high model order, low SNR), CSST requires numerous random starts to achieve high confidence that a global minimum to the reduced Chi-square function has been found. For instance, a six-dipole model of MEG visual data may require 15,000 random starts with a CSST analysis. By incorporating the results of a MUSIC analysis into the selection of starting dipole configurations in CSST, the number of starting configurations needed to obtain accurate CSST results can be greatly reduced. The MUSIC-seeded CSST (MS-CSST) algorithm performs one MUSIC analysis on a variable density grid of approximately 7000 locations. A novel clustering approach is used to find the MUSIC peaks, saving the best 20 dipole locations in each cluster for use with CSST. Various sampling strategies can be used when mixing in these MUSIC dipole locations with randomly selected locations, while creating the CSST starting dipole sets. We illustrate results for a five-dipole MEG simulation, based on an actual subject/sensor configuration with a Neuromag 122 system. The dipoles were given highly overlapping timecourses, with low signal to noise, so that the MUSIC procedure would not localize all of the dipoles well. A 5-dipole MS-CSST analysis was performed, using 100 pseudorandom initial dipole configurations, creating 50 sample configurations with 2 MUSIC locations and 3 random locations and 50 samples with 1 MUSIC location and 4 random locations. The same data was analyzed with CSST with no MUSIC seeding, using 1000 initial dipole configurations and saving the best 100 based on reduced Chi-square values. We compare the reduced Chi-square results for these two analyses in Fig. 9. Comparing the best 26 fits from each analysis, MS-CSST outperformed regular CSST, with one tenth as many initial dipole configurations.

Fig. 9
figure 9

This figure compares the performance of MS-CSST to regular CSST, showing the best 36 reduced Chi-square values for a 5-dipole analysis of MEG data generated from 5 simulated sources. MS-CSST slightly outperformed regular CSST, when comparing the best 26 solutions from each analysis, using one-tenth as many starting dipole sets

Full size image

In analyses of simulated MEG data, MS-CSST outperforms CSST with 90% fewer initial dipole configurations. In analyses of empirical MEG data, the performance improvement has been less, requiring 75–80% fewer starting configurations with MS-CSST versus CSST to produce comparable results for the best 5 to 10 solution sets.

6.3 MEG/EEG Signal Processing and Data Organization Using MEGAN

All of the human subject MEG/EEG data used by CSST are first processed using MEGAN, a software package written in IDL by Elaine Best (1998); MEGAN provides a large suite of tools to process and view MEG/EEG sensor data. The signal processing tools provided include noise filtering, artifact rejection, pass averaging, and other capabilities needed to work with MEG/EEG data. MEGAN is graphical user interface base and provides several interfaces for viewing raw and processed data, including overlaid multisensor signal time plots, signal plots on a map of the sensor locations, and colored contour plots (and movies). MEG/EEG data sets processed by MEGAN are written to a standardized format, called netMEG, which is based on netCDF. It is thus a self-describing format that can be accessed using standard libraries available for several major programming languages, including C and Fortran. Using these netMEG files to provide the MEG/EEG input data for CSST greatly simplifies reading in and setting up the data for CSST analyses.

6.4 Combining MEG and EEG

CSST provides a method for analyzing combined MEG and EEG data. This is done using the same nonlinear simplex analysis as is used for a single-modality analysis, but averaging the MEG and EEG reduced Chi-square values at each step of the simplex search. The user sets a weighting that determines the relative contribution of each modality to this combined metric. Only preliminary work has been done on determining what weighting value to use for a given combined analysis. Criteria to consider include the relative SNR of the MEG versus EEG data being analyzed (usually higher for EEG), the relative overall quality of the data (e.g., sensor count and spacing), and the fact that localization accuracy is generally better for MEG than EEG.

6.5 Combining fMRI/MEG/EEG

CSST can be used to perform combined analyses of MEG and/or EEG with fMRI or other volume-based brain imaging data, using the concept of a likelihood volume (LV). An LV is used in CSST to weigh the reduced Chi-square values of the multi-dipole analysis using information from other functional (or volume-based) brain imaging modalities, such as fMRI, PET, or structural MRI. Voxels in the LV contain a value in the interval [0, 1], representing the likelihood of cortical activity at that voxel location. If V is the value in a voxel of the LV, an error value, VE is obtained using VE = 1.0 − V. If AVE is the average of the VE values for a set of dipole locations, CE is the reduced Chi-square value for this dipole set, and if W is a user-selected weighting value, then the new error measure for the simplex search is defined by E = W ∗ CE + (1 − W) ∗ AVE. This error measure will influence the simplex search for the best-fitting dipole location sets but still allows dipole locations in best-fitting sets that do not fall within regions of high likelihood, as determined by the LV.

As used with CSST, an LV is constructed with the same resolution as the structural MRI volume and is registered to this MRI volume and, thus, the MEG or EEG data. In the case of BOLD fMRI, an LV can be obtained by thresholding and scaling the T-statistic volume derived from the fMRI analysis to [0, 1] and then resampling and registering this volume to the subject’s sMRI volume.

A simulation-based analysis of the use of combined MEG/fMRI for analyzing single-pass MEG data is presented below. That analysis shows the possible benefits of using a combined MEG/fMRI analysis. The use of a LV has more recently been incorporated in a Bayesian combined fMRI/MEG analysis procedure (Jun et al. 2008).

7 Forward Simulator

To aid incomparing alternative MEG/EEG inverse algorithms using more realistic current distributions than single or multiple dipoles, a focal- and distributed-source Forward Simulator was developed within the MRIVIEW framework. Using the MRIVIEW surface viewing mode, ellipsoidal regions of the g/w matter boundary can be labeled. A user-specified oriented ellipsoid is used to create simulated regions of activity that can, for instance, lie along one bank of a sulcus. Multiple regions of arbitrary size and orientation can be created. Figure 10 shows three regions created using the Forward Simulator.

Fig. 10
figure 10

The Forward Simulator and constrained 3D Interface were used to create three patches of simulated cortical activity

Full size image

The simulator interface contains a table, with a row to control the activation characteristics of each region. The activation timecourses can be modeled using multiple Gaussians or a sinusoid, or they can be read from a file. For each region, a maximum timecourse current is set using table entries for either current density or total current.

To generate EEG or MEG forward solutions, sensor geometries and other information are obtained from a netMEG file produced by the program MEGAN, usually from a human EEG or MEG study. Treating the cortical voxels in a region as oriented dipoles acting in concert, a spherical EEG or MEG forward is calculated using the timecourse for that region. The EEG forward uses the Sun algorithm (Sun 1997); the MEG forward uses the Sarvas formula (Sarvas 1987). The simulated sensor measurements are obtained by summing the forwards for all of the regions. Spatially uncorrelated Gaussian noise with a desired standard deviation can be added to the forward measurements, as can real noise from MEG/EEG experiments. Figure 11 shows the Forward Simulator interface being used to create a complex, 5-active-region simulation, showing plots of the simulated cortical activity, and the forward timecourses based on subject/sensor geometries obtained from a study using a Neuromag 122 MEG system. The MEG/EEG Forward Simulator has been used in several MEG studies (e.g., Stephen et al. 2002) and is a key component of the MEG-SIM Portal project (Aine et al. 2012; Sanfratello et al. 2010; see chapter “MEG-SIM Web Portal: A Database of Realistic Simulated and Empirical MEG Data for Testing Algorithms” by Sanfratello et al. in this volume).

Fig. 11
figure 11

The Forward Simulator interface, showing a 5-active-region simulation. The upper plot shows the simulated timecourses; the lower shows the resulting sensor measurements

Full size image

8 Combining MEG and fMRI for Single-Pass MEG Analysis: A Simulation Study

In a typical MEG sensory response experiment, a single MEG pass measures the brain activity corresponding to a single application of a stimulus, such as the sounding of a tone. Usually, 100 or more passes are recorded for a given stimulus, and their average is used to perform location and timecourse analysis of the stimulus-induced brain activity. Because of the low signal-to-noise ratio, inverse analysis of single-pass MEG data is extremely difficult. In some cases, using information from other sources could aid in this analysis. The fMRI data in a combined MEG-fMRI experiment can complement the MEG data, in terms of providing additional brain activity location information, but it does not provide information on millisecond activity timecourses.

The MEG Forward Simulator in MRIVIEW was used to create three regions of brain activity, with activity timecourses similar to those shown in the upper right plot in Fig. 12 and MEG sensor timecourses shown in the two left plots. The upper left plot shows noise-free sensor timecourses, and the lower left plot shows the same timecourses with added simulated noise, giving a noise level typical of single-pass MEG data.

Fig. 12
figure 12

The MRI head data shown in 3D was used to create simulated regions of brain activity. The white regions represent simulated fMRI data. The three-dipole CSST solution for the low-noise case is shown with the larger arrows. For the high-noise analyses, the MEG-only solution locations are shown using spheres. The combined MEG, fMRI solution locations are shown using the smaller arrows

Full size image

A simulated fMRI volume was generated by applying a smoothing algorithm to the voxels of the three simulated regions, plus four additional regions having no electrical activity, to create brain activity maps similar to those obtained in an fMRI analysis. A LV was obtained for these regions by scaling the voxel values in the smoothed volume to the interval [0, 1]. This volume was registered to the MEG coordinate space. During the CSST search process, the voxel values of the LV were used to weigh the MEG-only Chi-square measure for a given set of locations, to produce a combined (simulated) MEG-fMRI measure. This fMRI weighting influences the simplex search in CSST but does not constrain the dipole locations to only regions of fMRI activity.

The lower right plots in Fig. 12 show the benefits that may be realized with a combined MEG-fMRI analysis. The combined MEG-fMRI analysis demonstrates the fMRI influence on dipole locations. In the combined analysis, the fitted dipole locations are pulled toward regions of high fMRI activity, counteracting the influence of the MEG noise on the dipole fits. The dipole timecourses arising from the combined CSST analysis of the noisy simulated MEG data more closely match the low-noise analysis results, as evidenced by the shape and peak amplitudes shown in green and blue. In the MEG-only case, the peak green dipole timecourse amplitude is 50% higher than it should be, because of complications in dipole fitting arising from the sensor noise. A similar problem can be seen with the blue dipole. For both the green and blue dipoles, the combined analysis timecourse results closely match the actual timecourses. This improvement in timecourse matching most likely arises from the fMRI contribution reducing the error in fitted dipole locations due to the influence of sensor noise.

A LV can also be constructed by performing a CSST analysis of averaged (low-noise) MEG data obtained from the MEG data, by deriving a LV from a Monte Carlo analysis (a post-processing option in CSST) of the CSST results. The derived LV will have peaks at locations where the Monte Carlo analysis is most densely clustered. In either case, having a LV to influence the CSST fitting procedure shows promise as an approach to single-pass analysis. In cases where MEG and EEG are acquired simultaneously, the improved SNR will be even more likely to produce reasonable single-pass results using a combined MEG/EEG/LV analysis. Since this could provide spatially localized signal (time) frequency information, the results of these single-pass analyses could be useful in performing reconstructions of cortical networks.

9 Conclusion

The MRIVIEW brain imaging software package provides an integrated environment for analyzing MEG/EEG data and visualizing the results on MRI anatomy. The major components of MRIVIEW provide a wide range of capabilities, allowing a user to read in and view MRI data, segment anatomical structures, obtain MEG to MRI transformations, and create MEG/EEG forward simulations. MRIVIEW includes CSST, which is used to perform multi-start, multi-dipole-based analyses of MEG/EEG data, with options for using MUSIC-based seeding or fMRI data to improve minimization performance. Solutions from fMRI analyses or averaged multi-pass MEG data can also be used to create LVs. The use of these LVs in CSST shows promise as a means of obtaining dipole location and timecourse information from single-pass MEG/EEG data.

10 Obtaining MRIVIEW

MRIVIEW is freely available from this website: ftp.lanl.gov/public/ranken/mriview. The source code is also freely available for academic research use. The software is written in IDL and requires IDL to be used. A free version of IDL, called the IDL Virtual Machine, is available from ExelisVIS at www.ExelisVis.com. The Virtual Machine can be used to run third-party, GUI-based packages, such as MRIVIEW, but cannot be used for code development.