Introduction

With the evolution of functional magnetic resonance imaging (fMRI), the need to visualize statistical, neuroanatomical and functional images has increased. Although tables of clusters and coordinates provide a reliable way of ensuring that the results remain comparable, visualizations are an important way to convey the location and spatial extent of effects in functional neuroimaging studies. Traditionally, the human brain is visualized in three two-dimensional orthogonal planes (coronal, sagittal and axial projections) in order to show areas of increased activation or deactivation under certain conditions. However, such illustrations cannot convey the three-dimensional complexity of activated areas at a glance, particularly, in time-resolved depictions of functional activation changes of the whole brain. Therefore, novel applications are needed that facilitate a better understanding of the three-dimensional shape, distribution and time-dependent changes of brain areas.

In order to address this issue, some software packages provide a three-dimensional view of anatomical and functional MR-data. MRIcro (www.mricro.com) is one of the most popular viewers of medical images that is freely available (Rorden et al. 2007). Voxeldata can be rendered as a surface or volume object and statistical maps can be used as overlays. Other packages, such as 3DView (http://www.rmrsystems.co.uk/volume_rendering.htm), Slicer (Gering et al. 2001), DataViewer3D (Gouws et al. 2009) or fMRIVis (Rößler et al. 2006) offer similar capabilities.

These packages provide helpful functionality for displaying three-dimensional voxel volumes of anatomical and functional data. For instance, MRIcron depicts slices of anatomical MRI data along with functional overlays. These overlays can also be rendered as 3d-projections on a non-transparent voxel volume of the human brain. DataViewer3D is a recently published viewer for multi-modal neuroimaging data that depicts structural and functional MRI data as well as DTI and MEG/EEG data (Gouws et al. 2009). It supports 2d- and 3d-projections and the combination of data from different acquisition techniques. However, no software tool provides the ability to render animated functional and statistical, time-resolved MRI data and can combine them along with supplementary materials. Spectus3D (http://netedge.co.uk/s3d/) provides the functionality to display hemodynamic changes, but this application is apparently unavailable at present.

We present a toolbox for the widely-used Matlab application SPM (Friston 2008) which allows the computation and visualization of time-resolved functional neuroimaging data in the free 3d-software Blender (www.blender.org) (Hess 2007). We refer to this application as BrainBlend. BrainBlend offers algorithms for the computation of relative BOLD activation changes. The increase and decrease of BOLD activity can be computed as relative difference to baseline images or as relative difference to the preceding image. Thereby, the temporal effect of perceptually and cognitively demanding tasks can be investigated on individual and on group level. The resulting images can be saved in the Nifti-format (http://nifti.nimh.nih.gov/nifti-1/) for further analysis in SPM and other fMRI tools. Furthermore, BrainBlend allows the export of the results to the new voxeldata format for the voxeldata renderer of Blender, which has been developed to visualize volumetric objects through transparent voxeldata. With the new voxeldata renderer the computed hemodynamic changes can be visualized for the entire brain through voxel volumes, allowing a direct and intuitive view on the changes occurring in the brain. The usage of Blender for visualization enables the creation of high-quality pictures and animations of the human brain and hemodynamic responses. Supplementary materials can be integrated in the animation to view hemodynamic changes along with the experimental design and the presented stimuli, for instance. Additionally, any anatomical, functional and thresholded statistical parametric map in the Nifti-format can be exported with BrainBlend to Blender for the creation of comprehensive and illustrative 3d-images and animations of hemodynamic and statistical effects.

In contrast to other software packages, BrainBlend represents a toolbox that combines and supplements the statistical analysis capabilities of SPM with the visualization capabilities of the universal open-source 3d-software Blender. This application has been widely used in neuroimaging and molecular biology for the creation of visual stimuli (Creem-Regehr et al. 2007; Eger et al. 2008; Engel et al. 2008; Tavares et al. 2008), as a 3d-engine for a virtual-reality platform for clinical psychology and behavioral neurosciences (Gatti et al. 2008; Riva et al. 2007), for three-dimensional modeling of MRI data (Ventura et al. 2008), and for the visualization of scientific data (Heider et al. 2009; Morris et al. 2007; Nägerl et al. 2007; Palombi et al. 2006; Rivera-Calzada et al. 2005). In bioinformatics, the high demand for visualization techniques has already led to the formation of a new scientific field, the so-called “bioimaging informatics”, in which, among other software packages, Blender is used for visualizing molecular models (see Review by Peng 2008). Blender was originally designed to provide a powerful and sophisticated environment for the production of 3d-pictures and animations. Beside 3d-modeling techniques, this tool enables advanced compositing and post-production.

The combination of BrainBlend for the computation and export of Nifti data to Blender and Blenders 3d-capabilities allow scientists to visualize time-resolved BOLD activation changes for block- and event-related experimental settings, enabling a better and simultaneous understanding of hemodynamic activations in the spatial and temporal domain. In order to demonstrate the functionality of the toolbox, we used a freely available functional dataset acquired by Büchel and Friston (1997) to show how the temporal evolution of functional networks can be depicted.

The Application

Pictures and animations of fMRI images are generated in two steps. First, all necessary data are generated and converted to the voxeldata format using our SPM toolbox BrainBlend. A voxeldata file defines each voxel in terms of a certain property, which may be its density, emittance or color. It is possible to use the same voxeldata file for more than one property. However, the ability to define the same 3d-shape through more than one voxeldata file allows complex arrangements to convey information about a voxel through separate voxeldata files coding its color, density and emittance.

For data generation and export, we provide the plug-in BrainBlend that is integrated in the data analysis workflow of the Matlab software package SPM. With this application, the user can generate voxeldata that remain constant or alter their appearance in an animation. Thresholds, transformations, and scaling parameters can be defined for the export of the data in order to highlight certain areas or voxel contrasts.

BrainBlend for SPM5 is integrated into the batch user interface (UI), which is normally used to configure the preprocessing and statistics for the functional images (Fig. 1). In the so-called “voxeldata job” the user can create either a picture or animation for one Nifti- or Analyze-file or an animation for a series of Nifti-/Analyze-files. If only one fMRI image is used for the volumetric renderer (e.g. a certain functional network computed in the statistical analysis), the user can add a temporal dimension by importing a vector of intensity values from a text file, a regressor from a SPM.mat file, or by adding a vector manually. This is particularly useful for the visualization of spatial functional networks, extracted from EPI-image time-series by the independent component analysis (ICA) with their corresponding time course (Beckmann et al. 2005). After processing with ICA, the resulting Analyze- or Nifti-file contains normalized z-scores reflecting the degree to which a given voxel’s time-series correlates with the overall component time-series. Hence the function reads all types of Analyse- and Nifti-files, also statistical maps resulting from a T- or F-analysis can be used along with a time course. The generated voxeldata files contain the product of the intensity values multiplied by the image data for each point in time. This allows the user to create animations in which the intensity of activity can be visualized over time. In the final animation, several fMRI images can be equipped with separate time courses, visualizing the behaviour of functional networks.

Fig. 1
figure 1

User interface of the BEND-plugin in SPM5 to configure the analyze-/Nifti-export left: The user selects one image, specifies thresholding parameters and a time course. The Voxeldata-Exporter generates a voxelfile for Blender. right: A series of images can be used to generate an animation displaying relative changes (e.g. relative BOLD response) to a baseline

Full size image

Furthermore, a series of fMRI images can be utilized to visualize hemodynamic changes in contrast to a baseline. The baseline can be calculated from a subset of the image series or as relative contrast to a preceding image. Using a subset of the image series as baseline or control condition enables the user to visualize increased or decreased BOLD activation relative to a block of an experiment-related condition (e.g. rest or control task). The relative contrast to a preceeding image, however, highlights brain areas in which BOLD activation increases or decreases. It focuses more on the temporal change of BOLD activation than on the difference to a certain condition.

For the calculation of BOLD response changes according to a baseline, the volume data and their maximal relative changes are normalized voxelwise. Given that v defines the intensity of a voxel at a certain point in time of n images, its intensity for the voxeldata renderer v′ is calculated as follows:

$$ \begin{array}{*{20}{c}} {v\prime = {{\left( {\frac{v}{u} - a} \right)} \mathord{\left/{\vphantom {{\left( {\frac{v}{u} - a} \right)} d}} \right.} d}} \\ {u = \max \left( {{v_{1,}}{v_{2,}} \ldots, {v_n}} \right)} \\ {a = \frac{{{{\left( {\sum\limits_i^m {{v_i}} } \right)} \mathord{\left/{\vphantom {{\left( {\sum\limits_i^m {{v_i}} } \right)} m}} \right.} m}}}{u}} \\ {d = \mathop {{\max }}\limits_i \left( {\left| {\frac{{{v_i}}}{u} - a} \right|} \right)} \\ \end{array} $$
(1)
$$ \begin{array}{*{20}{c}} {w\prime = v\prime * s + t} \\ {w\prime = 0,\;{\text{if}}\;w\prime\kern1.5pt<\kern1.5pt{w_{\min }} \vee w\prime > {w_{\max }}} \\ \end{array} $$
(2)
$$ \begin{array}{*{20}{c}} {{\text{if}}\;w\prime > 0} \hfill \\ {\;\,m\prime = \min \left( {m + 1,{m_{\max }}} \right)} \hfill \\ {{\text{if}}\;w\prime = 0} \hfill \\ {\;\,m\prime = 0} \hfill \\ {{\text{if}}\;m\prime \ne {m_{\max }}} \hfill \\ {\;\,w\prime = 0} \hfill \\ \end{array} $$
(3)

Each voxel v is normalized by u, which represents the maximum value of all n points in time. This ensures that local signal differences do not affect the visualization of global activation changes. The normalized value v/u is subtracted by the normalized mean value a of the m dedicated baseline images. Since the percent of signal change in functional images is down to only 2–4% (Miezin et al. 2000), the relative difference between v/u and a is relatively low. Hence, the values can optionally be normalized by the maximal difference d of all n points in time (1). Additionally, the transformed voxel value v′ can be scaled and transposed by user defined values s and t to visualize deactivations or to intensify certain BOLD effects (2). The manually defined values w min and w max determine the range in which a BOLD activation difference should be displayed. This allows to mask all low differences and to highlight only activation differences that e.g. exceed a certain minimum (2).

To display only BOLD signal changes that survive more than one EPI image in the EPI time series, the user can specify m max , which indicates how long (how many images) a certain relative difference must exist until it is displayed in the resulting image. If w′ is greater than zero, the variable m is increased by one until it reaches m max , otherwise it is set to zero. If w′ is greater than zero for at least m max images (m′ = m max ), w′ will be rendered, otherwise it is set to zero (3). Additionally, BrainBlend uses a buffer to assure that these voxels will be rendered synchronously with the time line and not with a delay of (m max −1) pictures. The buffer is also used to guarantee that a voxel, that is greater than zero for at least m max images, is rendered for the whole period and not only for the moment it reaches m max . The results of the calculations are written to a new file, which can be used in Blender to render functional neuroimaging content and arrange it visually with experimental-related information and images.

Additionally, BrainBlend converts a serie of Nifti- or Analyze-images to the voxeldata-format without any computations or modifications. This function is useful, if statistical images, derived from a first- or second-level analysis should be rendered consecutively. Such animations can depict the increase of resource allocation on group level during different block-design conditions (e.g. different n-back conditions) or the temporal evolution of a BOLD response in an event-related design (e.g. using a finite impulse response approach for brain activation movies (Windischberger et al. 2008)).

The Voxeldata Renderer

The voxeldata renderer of Blender has been developed by one of the authors to meet the increased need for complex 3d visualization and flexible display of three-dimensional shapes and materials in 3d-animations. In neuroimaging, complex structures that cannot be adequately depicted by representing only their surface, such as anatomical MR and CT images or BOLD activity changes, are particularly suitable for a voxel-based renderer (Fig. 2).

Fig. 2
figure 2

Using a template for blender the user must only change the path to the voxeldata file. Advanced users of Blender can customize animation and rendering properties

Full size image

“Raycasting” is the reference algorithm in volumetric representation, as it uses physical based light interactions for rendering (Watt and Watt 1992). A rendered image is calculated by tracing each ray inside the volume region, taking into account the transparency properties of each voxel. The volumetric engine of Blender accounts for physically based light-interaction, like absorption (light gets absorbed and attenuated as it travels through a volumetric region), scattering (light gets dispersed due to interactions with particles in a volumetric region) and emission (the volume produces light), in a participating medium. Thus, almost any volumetric physical material can be represented. The voxeldata module is implemented as volumetric texture. Thereby, datasets imported by BrainBlend can be arranged with other datasets or texture types, such as procedural textures, allowing an enormous level of postprocessing.

Results

Anatomical Data

Anatomical data can be exported from SPM using BrainBlend in a straightforward manner by converting an Analyze- or Nifti-image to the voxeldata format. Density and color values can be adjusted subsequently in Blender. We converted the canonical T1-weighted image of a representative male individual, supplied with SPM96, with a minimum threshold (wmin) of 0.0, 0.2 and 0.4. Figure 3 shows the exported T1-template as transparent voxeldata. Additionally, voxeldata can also be rendered as solid cover, as illustrated with an MNI template (Fig. 4).

Fig. 3
figure 3

T1-image of SPM with light absorbing voxels. Lower boundaries of viewable voxels are >0.0, >0.2 and >0.4 (from left to right)

Full size image
Fig. 4
figure 4

Solid voxeldata of an MNI template

Full size image

Functional Data

Functional and anatomical data can be combined to display functional activation of the human brain. Since all data are transparent, the functional anatomy of an activated area can be grasped at a glance. Furthermore, views from different angles and animations can be created in Blender. In Fig. 5, activated (red) and deactivated (blue) areas are visible. To create such images, three voxeldata sets have to be exported: the anatomical image of the brain and the two desired thresholded z-maps extracted by an ICA, one for activation, one for deactivation. In order to display the activity of a functional network over time, the data can be multiplied with a time-course vector, representing the magnitude of activation of a network in the acquired images.

Fig. 5
figure 5

Two sets of functional data combined with anatomical data. Four different views of the data are presented

Full size image

Animation of an EPI Series

Sequences of EPI images can be used to visualize relative changes of the BOLD signal over time. To illustrate task-dependent relative BOLD-changes, we exported a dataset acquired by Büchel and Friston (1997) to the voxeldata format. The dataset was part of a study in which the influence of the psychological variable attention on the visual pathways was investigated (Büchel and Friston 1997). The study was structured in three conditions, representing the observation of dots (“photic”) that are stationary or moving (“motion”) and the velocity discrimination task (“attention”). For a full description of the study design see (Büchel and Friston 1997).

We used the dataset of Büchel and Friston to visualize positive BOLD-changes. The results can be downloaded at http://brainblend.sourceforge.net. The average of the first ten images was used as the baseline and subtracted from all other EPI images. Additionally, all relative BOLD changes were normalized as described above. Figure 6 shows the relative difference from the EPI time series images 9–14. Since every animation in Blender can be post-processed and rearranged with other contents, relative BOLD activation changes can also be displayed with the paradigm matrix and the presented stimuli, simultaneously (Fig. 7). This offers a new and more intuitive way to understand the data presented by Büchel and Friston.

Fig. 6
figure 6

Sequence of EPI activation differences embedded in a T1-image. The green grid shows the orientation of the head

Full size image
Fig. 7
figure 7

Animated voxeldata and study related material such as the paradigm matrix or the stimuli, can be arranged in Blender to track the BOLD response changes and the presented stimuli simultaneously. The green line (top right) indicates the current values of the regressors during the experiment. The stimuli (bottom left) were re-created according to Büchel and Friston (1997)

Full size image

Another advantage of BrainBlend is the fact that the animation can be used to visualize scanner-related artifacts in a scan session, or any other temporal effects that are hard to detect in the classical GLM analysis if they do not persist for a whole block of an experimental condition. The calculated, relative BOLD-activation changes can also be saved as Analyze images. Thus, the temporal emergence of functional networks can be explored using standardized coordinates and tools (Fig. 8).

Fig. 8
figure 8

Relative BOLD activation changes can also be saved as Analyze files and explored with standard tools like SPM. The illustrated image corresponds to image 6 of Fig. 6

Full size image

The availability of neuroimaging data in Blender offers far more possibilities that go beyond the scope of this paper. Figure 9 shows a collection of visualizations that can be achieved using BrainBlend for the computation and Blender for the rendering. For each of these visualizations, we provide templates for Blender with preconfigured settings. The user just has to replace the paths to the data to achieve the same results.

Fig. 9
figure 9

A collection of visualizations that can be achieved using BrainBlend for the computation and Blender for the rendering. Using solid voxels for the anatomy highlights activation changes in the cortex. Transparent voxel depict all activation changes at a glance. Animation of the data supports the three-dimensional location of the data

Full size image

Discussion

With the advent of neuroimaging, such as functional magnetic resonance imaging (fMRI), the need for visualization of statistical maps has increased considerably. We present a novel application called BrainBlend enabling further understanding and advanced visualization of the three-dimensional shape and distribution of activated areas. BrainBlend is a toolbox for SPM to convert Nifti- and Analyze-images to voxeldata files that can be rendered by Blender. The computation of voxeldata files is therefore integrated in the conventional fMRI analysis and can be embedded in any batch script written for SPM. Sequences of EPI-images and external time-courses can be used to create animated voxeldata, enabling the depiction of the temporal behavior of functional networks relative to a baseline. These animations provide a fast and effective way to compare the design of a general linear model with relative activation changes in fMRI data and to display interactions between independent networks in the temporal domain. Furthermore, any type of Analyze-/Nifti-image can be displayed as transparent or solid voxeldata to facilitate imagination of the three-dimensional shape of certain anatomical or functional structures.

In order to demonstrate the above-mentioned characteristics, we applied our toolbox both to sample data provided by SPM and to a subject measured in a study conducted by Büchel and Friston (1997). We extracted the temporal evolution of relative BOLD activation changes and used BrainBlend with the new voxeldata renderer of Blender for visualization. Animations of voxeldata have been created to render a time-resolved depiction of hemodynamic activity. A time course, extracted by independent component analysis and relative BOLD activity changes calculated by BrainBlend, provided the necessary transparency values. Visualization of the experimental stimuli and the paradigm matrix has been added to the resulting animation. Images of relative BOLD activation can also be saved as Analyze- or Nifti-images to track regions of interest with SPM or any other software. The computed data can be visualized in various ways as shown at the end of the results-section. All presented pictures and supplementary material, such as animations of relative BOLD activation changes, are available at http://brainblend.sourceforge.net.

An important difference between software packages such as FSLView, MRICron, 3DSlicer or DataViewer3D and BrainBlend is that BrainBlend represents a toolbox for SPM and an interface between SPM and Blender while the other products represent stand-alone applications. The programs have been designed to meet the specific visualization needs of neuroimaging data, while BrainBlend makes neuroimaging data available for a universal 3d-software. This circumstances imply some advantages and disadvantages. The mentioned programs are suited for the detailed depiction and anatomical location of statistical data, through slice-views of the brain (like offered by each of the mentioned programs) and polygonal renders of neuroimaging data (as used in FSLView and DataViewer3D). Users that are familiar with Blender can achieve the same results but the time and effort is much higher, since Blender has not been explicitly designed for this purpose. Voxel-based render-engines are supported by MRICron and 3Dslicer, but do only allow the depiction of one dataset as voxel volume at a time (3DSlicer) or are limited concerning their quality and flexibility (MRICron) compared to Blender. As FSLView, MRICron, 3DSlicer and DataViewer3D are written for neuroscientists, they provide directly access to voxel values and MNI or Talairach coordinates which is not provided by Blender so far, although absolute voxel coordinates can be determined.

BrainBlend, however, is a toolbox for SPM that fullfills mainly two purposes that are not provided by other software packages. First, BrainBlend computes relative BOLD activation changes across time in the entire brain. This is a feature that is not supported by other programs although programs such as SPM, FSL and even MRICron allow the extraction of BOLD time-courses of selected volumes of interest for further analyses. Second, it converts anatomical, functional and statistical (e.g. contrast- or parameter estimation) images to a format that allows its depiction as voxel volumes in Blender in which a voxelrender-engine has been implemented. As illustrated above, this allows far more flexibility in the arrangement with other voxel volumes and 2d- and 3d-content as Blender is inherently written for that purpose.

Like DataViewer3D, BrainBlend is written in an interpreter language allowing direct access to the code without the need of recompilation (3D Slicer offers an interface for extensions). Since BrainBlend is written in Matlab using the plugin interface and functions of SPM, a user who is familiar with SPM and Matlab can easily adapt and extend the presented functions for their own use or embed a BrainBlend job in the usual SPM5 or SPM8 analysis routine. The integration of BrainBlend into SPM as toolbox made the invention of a new software architecture dispensable. Any feature provided by BrainBlend is encoded in a single m-file using some helper functions of SPM, which makes it easy to access and modify the code. Table 1 summarizes the differences between BrainBlend and other visualization tools.

Table 1 Feature summary of BrainBlend and Blender’s voxeldata renderer and comparison with four commonly used imaging data visualization tools: FSLView, MRICron, 3DSlicer and DataViewer3D
Full size table

For historical reasons, the user interface of Blender does not meet the design principles that can be found in classical windows or linux applications (unique toolbars and shortcuts, locations for certain functions, etc.). That means that simple modifications in the 3d environment are indeed easy to accomplish. but require knowledge of certain shortcuts, mouse actions and design rules of Blender which leads to a relatively steep learning curve for beginners. In order to make our presented tools available for SPM users who have never used Blender before, we created templates for Blender in which these details are appropriately preconfigured. The user just has to select the voxeldata files that he created with BrainBlend. Subsequently he can render a picture or animation. Alternatively he can open a preview window, which allows the exploration of the data in nearly realtime. Figure 10 shows the necessary steps that the user has to accomplish in order to get a result with the lowest possible effort. BrainBlend includes templates for the visualization of anatomical structures, of relative BOLD differences, of paradigm related material along with relative BOLD differences and many other templates showing different kinds of visualizations.

Fig. 10
figure 10

A template of BrainBlend in Blender. The user selects the type of dataset, he wants to edit, the color of the dataset and the path to the voxeldata. Subsequently a picture or an animation can be rendered. Experienced users can define a colorband instead of a single color for the dataset and arrange camera position and animation manually

Full size image

Since Blender is a highly sophisticated software to create 3d images and animations in general, it represents a universal tool for all needs of visualization. Its compositor and sequence editor allows the production of images and animations in one application. The high resolution and quality, in which pictures and animations can be rendered, makes our presented solution particularly useful for high quality publications, presentations, or educational material (e.g. videos in HD-quality). Our application BrainBlend and Blender are available under the Creative Commons License and are open source.

For time-resolved BOLD activation movies either the method provided with BrainBlend or functional images that are calculated using other approaches, as suggested by (Bagarinao et al. 2003; Cox et al. 1995) or (Windischberger et al. 2008), can be used. This makes BrainBlend particularly suitable for real-time fMRI simulations using existing and upcoming methods.

So far, the relative BOLD changes are computed as difference between a predefined baseline and a given functional image. In future versions we intend to develop and implement statistically based methods that allow the automated computation of significant relative BOLD changes on group level. So far, this has to be done manually by the user. Since all important configurations of Blender can be controlled with scripts, it is conceivable that Blender is called directly from SPM, rendering a picture or animation, without any human intervention. BrainBlend could just execute the Blender binary file via Matlab, along with a script that inserts the created voxeldata files into a selected template and starts the rendering process.

The voxeldata features of Blender are currently not in the official development branch of Blender, but can be downloaded as experimental version. However, the experimental version is stable and provides all features presented in this article. Our experimental version is currently under development and will find its way to the official development branch as soon as the revised Blender 2.5 version will be released.

In summary, we presented a solution to utilize MR images in general and BOLD activation changes in particular in a sophisticated—but free—3d animation software. Neuroimaging data can be visualized as volume voxel and combined with any other 2d- and 3d-content for scientific and illustrative purposes.