Abstract
Fiber tracking is a common method for analyzing 3D tensor fields that arise from diffusion-weighted magnetic resonance imaging. This method can visualize, e.g., the structure of the brain’s white matter or that of muscle tissue. Fiber tracking results in dense, line-based datasets that are often too large to understand when shown directly. This chapter provides a survey of recent illustrative visualization approaches that address this problem. We group this work into techniques that improve the depth perception of fiber tracts, techniques that visualize additional data about the tracts, techniques that employ focus+context visualization, visualizations of fiber tract bundles, representations of uncertainty in the context of probabilistic fiber tracking, and techniques that rely on a spatially abstracted visualization of connectivity.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
1 Introduction
The visual representation of brain connectivity (e.g., Margulies et al. [43]) is an active research field within visualization. This work has lead to numerous techniques [51, 52] to explore and better understand the connections in the brain. Many of these approaches are based on diffusion-weighted magnetic resonance imaging (dwMRI) [37, 67] which yields estimates for the directional diffusion of water at each of the sampled locations. These datasets are typically represented as simplified 3D tensor fields (DTI; e.g., Kratz et al. [39]) and one fundamental visualization technique is the depiction of fiber tracts extracted from these tensor datasets (e.g., Behrens et al. [4] and Mori and van Zijl [45]). One sub-field within fiber tracking is deterministic fiber tracking. In this case a fiber tract is only extended along one direction—the diffusion tensor’s principal eigenvector (e.g., see Fig. 1 as well as Mori and van Zijl [45] and Zhang et al. [66]). In contrast, probabilistic tracking approaches (e.g., Parker [48]) do not only follow a single direction when determining fiber tracts. Instead, they model the uncertainty of the tract direction at a location and follow several different directions at each sampled location. The results of this probabilistic tracking are typically presented visually as positional probabilities—rather than renderings of dense fiber tracts. Regardless of whether deterministic or probabilistic fiber tracking is used, however, the fiber tracking produces dense, line-based datasetsFootnote 1 that are often difficult to understand due to their overall structure and spatial organization.
The fiber tracts are either computed for the whole brain or based on local seeding. Typically, the individual tracts are depicted using lines (e.g., Zhukov et al. [69] and Zöckler et al. [71]) or shaded cylindrical tubes (e.g., Zhang et al. [66]). An additional color map is often applied in which the color shades represent the local orientation of the tracts to better understand the spatial character of the data (see Fig. 1). It is important to understand that these extracted fiber tracts do not show actual brain fibers. Rather, each fiber tract is an abstracted representation of a likely direction of many fibers in the brain. As one can see in Fig. 1, however, these depictions can be quite overwhelming and it can be difficult to understand the spatial structure and other aspects of the connectivity data—even for small selections of fiber tracts.
For this reason researchers have begun to explore the use of illustrative visualization techniques to improve the visual representations of fiber tract data. Illustrative visualization [53] is a sub-field of visualization that is inspired, in particular, by the methods and techniques used in traditional illustration—which has a centuries-long history of creating understandable depictions of scientific subject matter. For example, illustrative visualization can be used to better depict the spatial structure of 3D datasets, to free up visual variables for the depiction of additional aspects of the data, or to simultaneously show several layers of data. In this survey we review the different illustrative visualization approachesFootnote 2 that have been applied to DTI-based fiber tracking.Footnote 3 Specifically, we group the approaches according to whether they improve the depth perception of the dense fiber datasets, support the depiction of additional data such as uncertainty, enable focus+context visualizations, focus on abstracted fiber bundle representations, or use other forms of abstracted connectivity representations. This classification is not entirely unambiguous; however, it provides a useful structure and ambiguous cases are discussed appropriately.
2 Improving the Depth Perception of Fiber Tracts
In the majority of application scenarios, the visualizations generated based on the extracted fiber tracts are displayed as 2D projections using traditional PC-based workstations (in contrast to stereoscopic viewing environments such as CAVEs [14]). This means that, due to the denseness of the fiber tracts, it can become difficult to understand the spatial structure and the spatial relationship between different groups of fiber tracts. The problem can be alleviated somewhat by using shaded lines (e.g., Zöckler et al. [71] and Mallo et al. [42]) drawn with respect to a light source in the scene. When combined with line shadowing (e.g., Peeters et al. [49]), the results convey a much better sense of spatial structure. This basic approach or extensions of it have been used in cases when a complete, dense dataset of fibrous structure (such as muscle tissue ) needs to be visualized [15, 49, 68].
A related way to address the spatial perception problem is to represent each tract with a shaded tube (Fig. 1a)—in contrast to a non-illuminated line rendering (Fig. 1b).Footnote 4 While the shading (combined with directional color coding) assists depth perception to some degree, it presents a challenge: Tubes with a larger diameter improve depth perception but also produce overlaps, reducing amount of visible detail. It is thus difficult to strike a good balance between the need for detail and the need for depth perception (even with additional measures such as tube halos [64]).
In a first attempt to address this issue, Klein et al. [38] proposed to remove the tube shading and, instead, to use distance-encoded contours and shadows for the tubes. This approach is realized using several rendering passes and enhances depth perception by visually grouping similar, neighboring fibers and using shadows as a visual cue for depth ordering. Inspired by this early work, Everts et al. [24] presented a depth-dependent halo technique that used line-based triangle strips rather than tubes as the main primitive for depicting fiber tracts. Footnote 5 This decision was based on the observation that line-based techniques are able to show more detail than tube-based renderings (see the comparison in Fig. 1a, b). However, because using only lines does not provide good support for depth perception, they resort to using halosFootnote 6 to enhance depth perception. To prevent halos from overlapping neighboring fiber tracts, Everts et al. [24] apply them in a depth-dependent fashion by rendering view-aligned triangle strips (with separate line and halo components). These triangle strips are folded away from the viewer such that halos only appear for larger depth discontinuities. Everts et al. combine this approach with line width attenuation based on the distance of the line segment to the viewer.
As shown in Fig. 2,Footnote 7 the approach not only visually emphasizes fiber bundles but also shows a clear depth layering of the bundles—without the bundles ever being explicitly specified. As shown in Fig. 2, the depth-dependent halo technique works best for locally seeded selections of fiber tracts (Fig. 2a). In contrast, the spatial structure of whole-brain tractograms (Fig. 2b) is less clear due to the overlapping of the dense fiber tracts. The reduced co-linearity of the white matter fibers close to the gray matter of the brain further reduces clarity (filtering the dataset based on the fractional anisotropy (FA) value of the tensors can alleviate this problem somewhat).
This limitation of depth-dependent halos results from its very local approach for emphasizing spatial structure: it is only possible to provide halos and emphasize depth within the scope of individual line strips. Therefore, follow-up work has investigated more global approaches to assist people in perceiving the three-dimensional structure of fiber tract visualizations. These newer techniques are inspired by global illumination models in computer graphics, specifically ambient occlusionFootnote 8 [70].
A first technique in this group of approaches, presented by Díaz-García and Vázquez [17, 18], uses ambient occlusion halos around the fiber tracts. These halos are either computed in screen space (called SSAO halos) or in a geometry-aware fashion (called GA halos) and are combined with a depth-modulated line style (see the examples in Fig. 3). The screen-space ambient occlusion [44] essentially computes a local estimation of shadowing, while geometry-aware halos use a multi-pass approach that renders enlarged fiber tract geometries to generate halos in object space to simulate ambient occlusion. Díaz-García and Vázquez [17, 18] conclude that SSAO halos are best suited for large, dense datasets, while the GA halos are better for fiber tract selections which are more sparse.
Eichelbaum et al. [22] improved upon these approaches with line-based ambient occlusion technique called LineAO. Like Díaz-García and Vázquez’s work [17, 18], this approach uses screen-space ambient occlusion . Based on an in-depth discussion of the sampling theory behind the technique, however, Eichelbaum et al. handle both local detail and global structures by separating the computation of the line-based ambient occlusion from that of the local illumination. Eichelbaum et al. thus not only produce grayscale visualizations (e.g., in Fig. 4a) but can also combine the method with illuminated lines or shaded tubes. These combinations can use, for example, the established color coding based on the local line segment direction (e.g., Fig. 4b).Footnote 9
Figure 4 demonstrates how the LineAO visualization is able to convey a whole-brain tractogram with an excellent support of spatial perception, in contrast to depth-dependent halos (Fig. 2b). Using LineAO, local individual fibers, fiber bundles, and the global spatial structure are clearly visible. However, the LineAO technique is less-suited for more coarse datasets which do not provide enough occlusion to benefit depth perception. Therefore, when visualizing subsets of fiber tracts, the approaches by Díaz-García and Vázquez [17, 18] or Everts et al. [24] may perform better.
3 Visualization of Additional Data About Fiber Tracts
In typical application scenarios for DTI-based fiber tract visualizations, researchers and health professionals are not only interested in understanding the spatial structure of a dataset but are also interested in other aspects. It is thus important to understand how different visualization techniques support the display of additional data dimensions. Some illustrative visualization techniques specifically make additional visual variables available that can be used for data display.Footnote 10
One common additional property is the local orientation of the fiber tracts. In traditional visualizations, local orientation is typically displayed using a directional color coding (as previously shown in Fig. 1). Also depth-dependent halos [24] and LineAO [22] can easily be combined with this encoding (Figs. 5 and 4b, respectively).
Going beyond this simple color mapping, Jianu et al. [35] explore encoding local properties computed based on the DTI data by using color pairs which are then mapped onto tubular fiber tract visualizations in horizontal, vertical, diagonal, or diamond patterns. This visual mapping is intended to allow viewers to easily compare properties of spatially co-located fiber tracts.Footnote 11 Hermosilla et al. [30] also used a similar texturing approach to indicate directionality of fiber tracts (see Fig. 7).
Bundle names are another important piece if information when interpreting fiber tract visualizations. Such annotations are a common element in traditional illustrations and, hence, are also important for illustrative visualization. Due to the complex nature of fiber tract datasets and the long length and intertwining nature of the tracts, however, it is not possible to employ existing external annotation placement methods created in illustrative visualization. Petrovic et al. [50] address this challenge by not only providing an impostor-based rendering technique that allows them to visualize the fiber tracts with a high visual quality but that also allows them to map the bundle names directly onto the respective fiber tract representations (see Fig. 6).
An essential property of fiber tract data is that there is always some degree of uncertainty about a tract’s path. Depicting a tract using a discrete line or tube, however, suggests that it actually exists in that precise configuration. To encode several levels of uncertainty in fiber tract data, Hermosilla et al. [30] thus group fiber tracts based on each track’s uncertainty value and render them using different colors (using lower saturations for higher uncertainties). They combine this approach with screen-space ambient occlusion , texture patterns , and unsharp masking to improve the visualization’s visual quality. Two examples of this technique are shown in Fig. 7.
4 Focus+Context Visualization
In practical applications it is typically not sufficient to simply view and understand the structure of a fiber tract dataset by itself. Instead, viewers need to understand the spatial location of the fiber tracts with respect to anatomical landmarks such as the surface of the brain with its sulci and gyri. For this purpose researchers have developed a number of illustrative focus+context visualization techniques that combine fiber tract visualization with contextual rendering. Illustrative visualization approaches are particularly well suited for this purpose because they can make use of the illustration principles of abstraction and emphasis.
A simple form of focus+context visualization is the combination of traditional fiber visualization with an illustratively rendered brain surface (e.g., Berres et al. [5, 6] and Eichelbaum et al. [21]). Such brain surface representations that wrap around the fiber tracts typically employ illustrative selective transparency such that sulci are shown in a more opaque form and gyri are more transparent. This rendering style allows most of the focus to be visible though the surrounding context, while at the same time also providing the necessary landmarks for reference. Another interesting alternative for generating focus+context is to render a subset of the brain volume in a fashion that resembles an existing dissection method as done by Anwander et al. [1] and Schultz et al. [59]. Here, the a virtual form of a Klingler dissection provides the context, while fiber tracts in focus are rendered in front of it as can be seen in Fig. 8.
Other illustrative rendering approaches attempt to combine the fiber tract visualization with the context visualization in such a way that the visual styles of both elements of the visualization match—as in traditional illustrations in medical textbooks. For example, Svetachov et al. [61] combine the black-and-white fiber tracts rendered using depth-dependent halos [24] with a similarly black-and-white hatched visualization of the brain surface (see Fig. 9),Footnote 12 and use cutting planes and stippling to show regions of gray matter. The hatched context visualization of the brain surface is stylized using screen-space ambient occlusion and halos are added around the fiber tracts to help the two visualization layers integrate appropriately.
Other illustrative visualization techniques specifically aim to combine fiber tract visualizations of the brain’s structural connectivity with visualizations of other types of data to create illustrative multi-modal visualizations. For example, Born et al. [7] integrate tubular fiber tract visualizations with an existing illustrative visualization of functional brain connectivity data [33, 34]. The resulting visualization (see Fig. 10) uses cutting planes, color-coded regions of the gray matter in the opaquely visualized part of the brain and on the intersection surface, and a semi-transparent brain surface. This surface also shows brain activity obtained from functional MRI data. The semi-transparent context visualization employs illustrative techniques such as ambient occlusion , silhouettes , and halftoning .
Another approach by Schott et al. [57, 58] for illustrative multi-modal visualizations combines a direct volume rendering of MRI data as context with registered DTI fiber tracts as the focus. This visualization technique also uses ambient occlusion —not only within the volume data or within the fiber tracts individually, but between the two. This means that the volume generates occlusion for the tubular fiber tracts and vice versa. By using these illustrative effects together, Schott et al.’s [57, 58] technique nicely supports the perception of spatial depth of the depicted objects in their focus+context visualization (also see the approaches discussed in Sect. 2).
Rieder et al. [55] create a visualization that is visually similar to Born et al.’s [7]. They also combine the fiber tracts with an opaque and a semi-transparent rendering of the brain surface. The opaque rendering of the surface is produced using ambient occlusion and a texture-mapped cutting plane, while the transparent surface is rendered using silhouettes . Similar to Schott et al. [57, 58], they also ensure that the ambient occlusion effect of the fibers is not only applied to the fiber representation but also to the opaque volume rendering of the brain. Rieder et al.’s [55] visualization, however, is unique in that they combine their illustrative on-screen focus+context depiction with an interactive physical model that allows users to control the visualization. This physical model serves as a second layer of context that also provides an interaction proxy for exploring the visualization. In addition to adjusting the view on the visualization, users are also able to select specific fiber tract bundles (by touching one of the illuminated spots on the physical model).
Another recent focus+context visualization of brain anatomy and tractography data was presented by Reichenbach et al. [54]. In this visualization the authors focus on showing structural connectivity in the context of selected regions of interest and the general brain context. In both cases Reichenbach et al. provide context by emphasizing the cortex’s parcellation (see Fig. 11). Using illustrative techniques such as depth-dependent outlines they highlight the relationships between different regions of the brain, assisted specifically by showing structural connectivity. It is particularly interesting that this approach and the previously discussed one do not show connectivity information by rendering individual fiber tracts, but instead show abstracted fiber bundles. Such bundling approaches are discussed in detail next.
5 Visualization of Fiber Tract Bundles
As the last two examples show, there are some cases in which it is not necessarily important to observe each extracted fiber tract individually. Instead, these only need to show meaningful fiber bundles. Some of the previously discussed techniques show fiber bundles implicitly, for example depth-dependent halos [24] and LineAO [22]. These methods, however, rely on a visual identification of a bundle. To make it easier for viewers to identify bundles, researchers have developed methods to explicitly represent the bundles, for example through wrapped representations [23]. Here, illustrative visualization can help to combine the bundle representation with that of the individual fibers, to indicate the fact that the bundles are only abstractions and do not physically exist as dedicated objects in the brain, or to provide additional visual variables that can be used to display bundle properties such as confidence.
The initial concept of extracting a dedicated bundle surface from a set of related fiber tracts was extended by Röttger et al. [56] in their BundleExplorer tool. BundleExplorer uses semi-transparent bundle surfaces with silhouettes and explicitly represented internal fiber tracts. The tool also uses marker-dependent cutaway views to see through the bundle surface as well as bundle intersection highlighting. In this combination of rendering techniques, the internal fiber tracts serve as focus objects while the bundle surface and the intersections represent context information.
In a less traditional approach, Otten et al. [46, 47] use fiber tract clustering to derive a set of related tracts which they then depict using an illustrative rendering technique that does not actually need to derive an intermediary wrapped surface geometry. Instead, they still process all extracted fiber tracts individually for the visualization, but render each of them with a relatively wide halo they color based on the fiber bundle. This approach causes all fibers in a particular bundle to merge visually into a single shape (see Fig. 12).Footnote 13 To emphasize this effect, Otten et al. [46, 47] add an image-space contour around each of these colored shapes. Finally, to better indicate the directionality of the fibers within each bundle, they add what they call “hint lines”—a fiber tract rendering in which only the top-most lines remain visible due to the occlusion by the tract halos. By using different colors for each cluster, this technique allows viewers to get a good overview of the fiber bundle arrangement. The approach can also be combined with additional focus and context elements such as textured slices from volume data or explicit fiber tract visualizations.
6 Probabilistic Fiber Tracking and Uncertainty Visualization
Fiber tract bundles derived from DTI-based fiber tracking, however, are not clearly defined anatomical structures in the brain. This is due, in part, to limitations of the data acquisition process. Bundles are also uncertain, however, because the fiber tracking algorithm, in deterministic tractography, follows the direction of the tensor’s primary eigenvector until the anisotropy falls below a pre-defined threshold. That means that, depending on whether or not a fiber tract is located in regions with a large fractional anisotropy, it can have varying degrees of confidence or uncertainty .
These different levels of confidence can be illustrated on a per-fiber-tract-basis as shown by Hermosilla et al. [30] (see Fig. 3). The uncertainty, however, can also be visualized within a fiber bundle as demonstrated by Brecheisen et al. [11, 12] (see Fig. 13).Footnote 14 Instead of representing the confidence of each individual fiber, Brecheisen et al. group the fiber tracts of a given bundle into intervals of the same confidence range, render them with a halo like in Otten et al.’s [46, 47] work. They then integrate the resulting layers into a single visualization. Similar to the other approaches discussed before, they emphasize some or all of the levels with silhouettes for a better visual perception of the layers. They also use different schemes to indicate the decreasing confidence levels. These levels include warm-to-cool shading [29] (Fig. 13a), increasing color saturation (Fig. 13b), decreasing color saturation (Fig. 13c), decreasing opacity, light-to-dark shading (Fig. 13d), decreasing the amount of detail due to growing dilation around the fiber tracts, and an increasing amount of blur. The resulting bundle visualizations can also be combined with other data visualization techniques in focus+context views.
The problem of uncertainty in deterministic tractography can also be addressed by using probabilistic tractography (e.g., Parker [48]). As described at the beginning of the chapter, this technique does not follow a single, deterministic direction for each fiber tract integration step. Instead it follows several, depending on the probability that tracts follow a direction other than the tensor’s primary eigenvector.
Using techniques from illustrative visualization and inspired by traditional illustrations of brain connectivity, Goldau et al. [28] created a slice-based fiber stippling technique that shows the distribution of the probability field (see Fig. 14). The stipples are small line segment that are oriented along the main diffusion direction. They are created by projecting the diffusion vector onto the slice such that long, narrow stipples represent diffusion parallel to the cutting plane and short, wide stipples represent diffusion at an angle to the cutting plane. The stipples representing different bundles are shown using different colors. Hlawitschka et al. [31] extend this approach by employing a Poisson-Disk sampling to distribute the fiber stipples in order to ensure an adequate perception of the pattern on the slices. In both cases, fiber stippling (i.e., small oriented dashes) as an illustrative visualization technique is well suited to the visualization of probability data because many people intuitively associate dashes to uncertainty [9]. Thus the visualization metaphor employed by Goldau et al. [28] and Hlawitschka et al. [31] is easily understood by viewers. Goldau and Hlawitschka [27] also demonstrated that such visualizations can be integrated in multi-modal depictions of brain data. These multi-modal visualizations can use a variety of other data modalities such as MRI data, functional MRI data, or CT data to provide the necessary context (e.g., see Fig. 14b).
7 Spatially Abstracted Visualization of Connectivity
All the illustrative visualization techniques discussed so far are intended to represent the fiber tracts and the resulting connectivity data as faithfully to the anatomical data as possible. However, sometimes people are interested in understanding connectivity in the brain at a higher level, one at which the visual representation can deviate to some degree from the anatomy. Illustrative visualization is naturally an ideal candidate for such spatially abstracting visualizations.
Jianu et al. [36] describe a visualization approach in this category in which they create two-dimensional neural map representations in which bundles of similar fiber tracts are grouped and visualized using simplified line-based primitives.Footnote 15 Specifically, these simplified representations are derived by clustering and selecting the tract with the smallest distance to all tracts in a bundle, favoring longer tracts over shorter ones. The results are then rendered in an illustrative fashion (using tract silhouettes and a schematic brain volume projection as context ) on the sagittal, coronal, and transverse planes (see Fig. 15, bottom-left). These views are well established in medical practice and serve as a means to explore the correct locations of the corresponding fiber tracts in a linked 3D view. In an extension of this initial approach, Jianu et al. [36] also created an abstracted bundle representation of the clusters (see Fig. 15, right)—inspired by Holten’s [32] edge bundling approach. This abstract representation can serve as an alternative way of exploring the 3D fiber tract visualization (Fig. 15, top-left). Both approaches together demonstrate that the use of abstraction and illustrative depiction styles can support interactive exploration of more traditional visualizations of 3D data.
Jianu et al.’s [36] abstraction primarily supports the easy selection of previously computed fiber tract clusters or bundles since their approach removes the complexity of the dense line representations. Sometimes, however, it is also necessary to understand the inner structure of a complete set of fiber tracts (such as depicted in Fig. 1)—a goal that is not easily supported using traditional forms of depiction, even illustrative ones (e.g., Figs. 2b, 4, and 5). To address this problem, Everts et al. [26] describe an abstraction technique that spatially contracts full-brain fiber tract datasets based on their local similarity. This similarity is computed based on co-linearity within a neighborhood of a given scale level. The result of this processing is that fiber tracts locally contract perpendicularly to the tract direction, revealing the global structure of the brain’s white matter by creating volumetric voids (e.g., see Fig. 16). For contractions at scale levels ≤ 2 mm, Everts et al. [26] show that the fiber tracts stay within the bounds of their corresponding fractional anisotropy areas. The authors also discuss how—when using larger displacements—the contraction can lead to anatomically incorrect depictions but show that these can reveal the brain’s higher-level organization. As such their contraction relates to approaches that introduce significant distortions such as done by Correa et al. [16]. They used deformations of line data (including neurological fiber tracts) to provide insight into the complex structures by deforming their paths or visually separating subsets.
8 Summary and Conclusion
This chapter provided a survey of illustrative visualization techniques applied to tractography data that resulted from 3D tensor fields based on diffusion-weighted MRI. The survey showed that illustrative visualization is used, in particular, to improve the depth perception in complex visualizations, to facilitate the visualization of additional data such as tract names, tract confidence, or similarity between tracts, to combine fiber tract visualization with additional visual elements in focus+context visualizations, to visualize fiber tract bundles, and to facilitate further spatial abstraction, for example, for an interactive exploration of the data. Of course, such a classification is not necessarily exclusive: a technique that primarily aims for a focus+context visualization, for example, can also make use of techniques that improve the depth perception of the fiber tracts or of the whole visualization.
The techniques described in this paper, of course, closely relate to other illustrative visualization techniques in brain connectivity visualization and beyond. For instance, the structural connectivity that can be explored though deterministic and probabilistic tractography closely relates to functional connectivity that can be examined based on functional MRI data. An example for such a visualization of functional data was presented by Böttger et al. [8] who employ edge bundling and bundle-driven transparency as forms of illustrative visualization to create representations of whole-brain functional connectivity (e.g., Fig. 17).Footnote 16 Of course, the tract-based depictions of brain connectivity also closely relate to streamlines, pathlines, etc. in fluid mechanics which can use similar illustrative visualization methods (e.g., Brambilla et al. [10], Everts et al. [24, 25], Li and Shen [41], and Shafii et al. [60]).
Naturally, the work surveyed in this chapter only represents a subset of the field of illustrative visualization, with explores methods to highlight, provide emphasis, or introduce abstraction [53]. When using these methods, however, we need to be aware of the implications of the general approach. While an illustrative visualization style can be attractive and can thus positively affect viewers [3, 65], it can also have a high “suggestive” power: Illustrative visualization may suggest information that can sometimes be misleading precisely because they often provide precise depictions and clear representations. Everts et al. [24], for instance, reported that neurosurgeons mentioned that fiber tract visualizations that use depth-dependent halos appear to depict the actual neuronal fibers (axons) of the brain—something that the data sources and the resulting visualizations do not provide. So while we should always be aware of these potential challenges for illustrative visualization, this example also illustrates the intriguing power of illustrative methods within visualization.
Notes
- 1.
This is also true for probabilistic tracking [4], even if probabilistic tractography results—due to the size of the generated data—are typically visualized by displaying the scalar probabilities that different brain regions are connected to a seed region. In fact, these dataset sizes are one motivation to employ illustrative visualization as it promises to present the data in an understandable form.
- 2.
- 3.
The chapter focuses on the visualization of brain connectivity. The discussed methods, however, can also be applied to other datasets that have similar characteristics, for example muscle fiber data.
- 4.
For a comparison of simple line rendering, shaded tubes, illuminated line rendering, and illuminated line rendering with shadowing see Figure 4 in Peeters et al.’s [49] paper.
- 5.
Everts et al.’s [24] approach could be viewed as an abstraction of line-based rendering with shadowing: it uses lines as the basic primitive, conveys occlusion, and does not rely on line shading.
- 6.
- 7.
The example images in Fig. 2 were created with the depth-dependent halos demo; see the project website at http://tobias.isenberg.cc/VideosAndDemos/Everts2009DDH .
- 8.
Ambient occlusion has also already been used in other sub-fields of visualization [62].
- 9.
The example images in Figs. 1 and 4 were created with the tool OpenWalnut [19, 20]; see the website at http://www.openwalnut.org/ .
- 10.
In the context of the brain connectivity visualization, Laidlaw et al. [40] have used this principle to illustratively show slices of DTI data based on inspirations from oil painting.
- 11.
Everts et al. [25] extended this idea, encoding data properties in (colored) patterns for flow data.
- 12.
The example images in Fig. 9 were created with the project’s demo; see the website at http://tobias.isenberg.cc/VideosAndDemos/Svetachov2010DCI .
- 13.
The example images in Fig. 12 were created with the vIST/e project’s demo (using a test release); see the website at http://bmia.bmt.tue.nl/software/viste/ and the SourceForge repository at http://sourceforge.net/projects/viste/files/ .
- 14.
The images in Fig. 13 were created with DTITool (provided by Ralph Brecheisen), an early version of the vIST/e software.
- 15.
A downloadable demo of Jianu et al.’s [36] technique is available at http://graphics.cs.brown.edu/research/sciviz/newbraininteraction/tutorial.htm and an online demo can be found at http://graphics.cs.brown.edu/research/sciviz/newbraininteraction/BrainComplete/P3/gmap_brain.html .
- 16.
The example image in Fig. 17 was created with the braingl tool, see the webpage at http://code.google.com/p/braingl/ .
References
Anwander, A., Schurade, R., Hlawitschka, M., Scheuermann, G., Anderson, T.R., Knösche, T.R.: White matter imaging with virtual Klingler dissection. NeuroImage 47(Suppl. 1), S105 (2009). doi:10.1016/S1053-8119(09)70916-4
Appel, A., Rohlf, F.J., Stein, A.J.: The haloed line effect for hidden line elimination. ACM SIGGRAPH Comput. Graphics 13(3), 151–157 (1979). doi:10.1145/800249.807437
Bach, B., Dragicevic, P., Huron, S., Isenberg, P., Jansen, Y., Perin, C., Spritzer, A., Vuillemot, R., Willett, W., Isenberg, T.: Illustrative data graphics in 18th–19th century style: A case study. In: Posters at IEEE Visualization (2013). Extended abstract and poster
Behrens, T.E.J., Sotiropoulos, S.N., Jbabdi, S.: MR diffusion tractography. In: Johansen-Berg, H., Behrens, T.E.J. (eds.) Diffusion MRI: From Quantitative Measurement to In-vivo Neuroanatomy, 2nd edn., Chap. 19, pp. 429–451. Academic Press, San Diego (2014). doi:10.1016/B978-0-12-396460-1.00019-6
Berres, A.S., Goldau, M., Tittgemeyer, M., Scheuermann, G., Hagen, H.: Tractography in context: multimodal visualization of probabilistic tractograms in anatomical context. In: Proceedings of VCBM, pp. 9–16. Eurographics Association, Goslar (2012). doi:10.2312/VCBM/VCBM12/009-016
Berres, A.S., Goldau, M., Tittgemeyer, M., Scheuermann, G., Hagen, H.: Tractography in context: multimodal visualization of probabilistic tractograms in anatomical context. In: Posters of IEEE BioVis (2012)
Born, S., Jainek, W.M., Hlawitschka, M., Trantakis, C., Meixensberger, J., Bartz, D.: Multimodal visualization of DTI and fMRI data using illustrative methods. In: Bildverarbeitung für die Medizin, pp. 6–10. Springer, Berlin/Heidelberg (2009). doi:10.1007/978-3-540-93860-6_2
Böttger, J., Schäfer, A., Lohmann, G., Villringer, A., Margulies, D.S.: Three-dimensional mean-shift edge bundling for the visualization of functional connectivity in the brain. IEEE Trans. Vis. Comput. Graphics 20(3), 471–480 (2014). doi:10.1109/TVCG.2013.114
Boukhelifa, N., Bezerianos, A., Isenberg, T., Fekete, J.D.: Evaluating sketchiness as a visual variable for the depiction of qualitative uncertainty. IEEE Trans. Vis. Comput. Graphics 18(12), 2769–2778 (2012). doi:10.1109/TVCG.2012.220
Brambilla, A., Carnecky, R., Peikert, R., Viola, I., Hauser, H.: Illustrative flow visualization: state of the art, trends and challenges. In: Eurographics State of the Art Reports, pp. 75–94. Eurographics Association, Goslar (2012). doi:10.2312/conf/EG2012/stars/075-094
Brecheisen, R., Platel, B., ter Haar Romeny, B.M., Vilanova, A.: Illustrative uncertainty visualization of DTI fiber pathways. In: EuroVis Posters (2011)
Brecheisen, R., Platel, B., ter Haar Romeny, B.M., Vilanova, A.: Illustrative uncertainty visualization of DTI fiber pathways. Vis. Comput. 29(4), 297–309 (2013). doi:10.1007/s00371-012-0733-9
Bruckner, S., Gröller, M.E.: Enhancing depth-perception with flexible volumetric halos. IEEE Trans. Vis. Comput. Graphics 13(6), 1344–1351 (2007). doi:10.1109/TVCG.2007.70555
Bryson, S.: Virtual reality in scientific visualization. Commun. ACM 39(5), 62–71 (1996). doi:10.1145/229459.229467
Chen, W., Yan, Z., Zhang, S., Crow, J.A., Ebert, D.S., McLaughlin, R.M., Mullins, K.B., Cooper, R., Ding, Z., Liao, J.: Volume illustration of muscle from diffusion tensor images. IEEE Trans. Vis. Comput. Graphics 15(6), 1425–1432 (2009). doi:10.1109/TVCG.2009.203
Correa, C.D., Silver, D., Chen, M.: Illustrative deformation for data exploration. IEEE Trans. Vis. Comput. Graphics 13(6), 1320–1327 (2007). doi:10.1109/TVCG.2007.70565
Díaz-García, J.: Rapid halo-based illustrative visualization of brain fiber tracts. Master’s thesis, BarcelonaTech/Universitat Politècnica de Catalunya, Spain (2011). doi:2099.1/14366
Díaz-García, J., Vázquez, P.P.: Fast illustrative visualization of fiber tracts. In: Advances in Visual Computing. Lecture Notes in Computer Science, vol. 7431, pp. 698–707. Springer, Berlin/Heidelberg (2012). doi:10.1007/978-3-642-33179-4_66
Eichelbaum, S., Goldau, M., Philips, S., Reichenbach, A., Schurade, R., Wiebel, A.: OpenWalnut: A new tool for multi-modal visualization of the human brain. In: Posters of EG VCBM (2010)
Eichelbaum, S., Hlawitschka, M., Wiebel, A., Scheuermann, G.: OpenWalnut – An open-source visualization system. In: Proceedings of 6th High-End Visualization Workshop, pp. 67–78. Lehmanns Media—LOB.de, Berlin (2010)
Eichelbaum, S., Wiebel, A., Hlawitschka, M., Anwander, A., Knösche, T.R., Scheuermann, G.: Visualization of effective connectivity of the brain. In: Proceedings of VMV, pp. 155–162. Eurographics Association, Goslar (2010). doi:10.2312/PE/VMV/VMV10/155-162
Eichelbaum, S., Hlawitschka, M., Scheuermann, G.: LineAO—Improved three-dimensional line rendering. IEEE Trans. Vis. Comput. Graphics 19(3), 433–445 (2013). doi:10.1109/TVCG.2012.142
Enders, F., Sauber, N., Merhof, D., Hastreiter, P., Nimsky, C., Stamminger, M.: Visualization of white matter tracts with wrapped streamlines. In: Proceedings of Visualization, IEEE Computer Society, Los Alamitos, pp. 51–58 (2005). doi:10.1109/VISUAL.2005.1532777
Everts, M.H., Bekker, H., Roerdink, J.B.T.M., Isenberg, T.: Depth-dependent halos: Illustrative rendering of dense line data. IEEE Trans. Vis. Comput. Graphics 15(6), 1299–1306 (2009). doi:10.1109/TVCG.2009.138
Everts, M.H., Bekker, H., Roerdink, J.B.T.M., Isenberg, T.: Illustrative line styles for flow visualization. In: Proceedings of Pacific Graphics, pp. 105–110. Eurographics Association, Goslar (2011). doi:10.2312/PE/PG/PG2011short/105-110
Everts, M.H., Begue, E., Bekker, H., Roerdink, J.B.T.M., Isenberg, T.: Exploration of the brain’s white matter structure through visual abstraction and multi-scale local fiber tract contraction. IEEE Trans. Vis. Comput. Graphics 21 (2015, to appear). doi:10.1109/TVCG.2015.2403323)
Goldau, M., Hlawitschka, M.: Multi-modal visualization of probabilistic tractography. In: Visualization in Medicine and Life Sciences III. Springer, Berlin (2015)
Goldau, M., Wiebel, A., Gorbach, N.S., Melzer, C., Hlawitschka, M., Scheuermann, G., Tittgemeyer, M.: Fiber stippling: An illustrative rendering for probabilistic diffusion tractography. In: Proceedings of BioVis, pp. 23–30. IEEE Computer Society, Los Alamitos (2011). doi:10.1109/BioVis.2011.6094044
Gooch, A.A., Gooch, B., Shirley, P., Cohen, E.: A non-photorealistic lighting model for automatic technical illustration. In: Proceedings of SIGGRAPH, pp. 447–452. ACM, New York (1998). doi:10.1145/280814.280950
Hermosilla, P., Brecheisen, R., Vázquez, P.P., Vilanova, A.: Uncertainty visualization of brain fibers. In: Proceedings of CEIG, pp. 31–40. Eurographics Association, Goslar (2012). doi:10.2312/LocalChapterEvents/CEIG/CEIG12/031-040
Hlawitschka, M., Goldau, M., Wiebel, A., Heine, C., Scheuermann, G.: Hierarchical Poisson-disk sampling for fiber stipples. In: Visualization in Medicine and Life Sciences, pp. 19–23. Eurographics Association, Goslar (2013). doi:10.2312/PE.VMLS.VMLS2013.019-023
Holten, D.: Hierarchical edge bundles: Visualization of adjacency relations in hierarchical data. IEEE Trans. Vis. Comput. Graphics 12(5), 741–748 (2006). doi:10.1109/TVCG.2006.147
Jainek, W.: Illustrative visualization of brain structure and functional MRI data. Master’s thesis, Eberhard-Karls-Universität Tübingen, Germany (2007)
Jainek, W.M., Born, S., Bartz, D., Straßer, W., Fischer, J.: Illustrative hybrid visualization and exploration of anatomical and functional brain data. Comput. Graphics Forum 27(3), 855–862 (2008). doi:10.1111/j.1467-8659.2008.01217.x
Jianu, D., Zhou, W., Demiralp, Ç., Laidlaw, D.H.: Visualizing spatial relations between 3D-DTI integral curves using texture patterns. In: IEEE Visualization Poster Compendium (2007)
Jianu, R., Demiralp, Ç., Laidlaw, D.H.: Exploring brain connectivity with two-dimensional neural maps. IEEE Trans. Vis. Comput. Graphics 18(6), 978–987 (2012). doi:10.1109/TVCG.2011.82
Johansen-Berg, H., Behrens, T.E.J. (eds.): Diffusion MRI: From Quantitative Measurement to In-vivo Neuroanatomy, 2nd edn. Academic Press, San Diego (2014). doi:10.1016/B978-0-12-396460-1.01001-5
Klein, J., Ritter, F., Hahn, H.K., Rexilius, J., Peitgen, H.O.: Brain structure visualization using spectral fiber clustering. In: ACM SIGGRAPH 2006 Research Posters, New York, p. 168(1) (2006). doi:10.1145/1179622.1179816
Kratz, A., Auer, C., Stommel, M., Hotz, I.: Visualization and analysis of second-order tensors: Moving beyond the symmetric positive-definite case. Comput. Graphics Forum 32(1), 49–74 (2013). doi:10.1111/j.1467-8659.2012.03231.x
Laidlaw, D.H., Ahrens, E.T., Kremers, D., Avalos, M.J., Jacobs, R.E., Readhead, C.: Visualizing diffusion tensor images of the mouse spinal cord. In: Proceedings of Visualization, pp. 127–134. IEEE Computer Society, Los Alamitos (1998). doi:10.1109/VISUAL.1998.745294
Li, L., Shen, H.W.: Image-based streamline generation and rendering. IEEE Trans. Vis. Comput. Graphics 13(3), 630–640 (2007). doi:10.1109/TVCG.2007.1009
Mallo, O., Peikert, R., Sigg, C., Sadlo, F.: Illuminated lines revisited. In: Proceedings of Visualization, pp. 19–26. IEEE Computer Society, Los Alamitos (2005). doi:10.1109/VISUAL.2005.1532772
Margulies, D.S., Böttger, J., Watanabe, A., Gorgolewski, K.J.: Visualizing the human connectome. NeuroImage 80, 445–461 (2013). doi:10.1016/j.neuroimage.2013.04.111
Mittring, M.: Finding next gen: CryEngine 2. In: ACM SIGGRAPH Courses, New York, Chap. 28, pp. 97–121 (2007). doi:10.1145/1281500.1281671
Mori, S., van Zijl, P.C.: Fiber tracking: Principles and strategies – A technical review. NMR Biomed. 15(7–8), 468–480 (2002). doi:10.1002/nbm.781
Otten, R.J.G.: Illustrative rendering of white matter fiber bundles. Master’s thesis, TU Eindhoven, Department of Mathematics and Computer Science, the Netherlands (2009)
Otten, R., Vilanova, A., van de Wettering, H.M.M.: Illustrative white matter fiber bundles. Comput. Graphics Forum 29(3), 1013–1022 (2010). doi:10.1111/j.1467-8659.2009.01688.x
Parker, G.J.M.: Probabilistic fiber tracking. In: Jones, D.K. (ed.) Diffusion MRI: Theory, Methods, and Applications, Chap. 23. Oxford University Press, Oxford (2010). doi:10.1093/med/9780195369779.003.0023
Peeters, T.H.J.M., Vilanova, A., Strijkers, G.J., ter Haar Romeny, B.M.: Visualization of the fibrous structure of the heart. In: Proceedings of VMV, pp. 309–316. Akademische Verlagsgesellschaft Aka GmbH, Berlin (2006)
Petrovic, V., Fallon, J., Kuester, F.: Visualizing whole-brain DTI tractography with GPU-based tuboids and LoD management. IEEE Trans. Vis. Comput. Graphics 13(6), 1488–1495 (2007). doi:10.1109/TVCG.2007.70532
Pfister, H., Kaynig, V., Botha, C.P., Bruckner, S., Dercksen, V.J., Hege, H.C., Roerdink, J.B.T.M.: Visualization in connectomics. Technical Report 1206.1428 [cs.GR], arXiv.org (2012)
Pfister, H., Kaynig, V., Botha, C.P., Bruckner, S., Dercksen, V.J., Hege, H.C., Roerdink, J.B.T.M.: Visualization in connectomics. In: Hansen, C.D., Chen, M., Johnson, C.R., Kaufman, A.E., Hagen, H. (eds.) Scientific Visualization, Mathematics and Visualization, pp. 221–245. Springer, London (2014). doi:10.1007/978-1-4471-6497-5_21
Rautek, P., Bruckner, S., Gröller, E., Viola, I.: Illustrative visualization: New technology or useless tautology? ACM SIGGRAPH Comput. Graphics 42(3), 4:1–4:8 (2008). doi:10.1145/1408626.1408633
Reichenbach, A., Goldau, M., Hlawitschka, M.: Combined three-dimensional visualization of structural connectivity and cortex parcellation. In: Proceedings of VMV, pp. 71–78. Eurographics Association, Goslar (2014). doi:10.2312/vmv.20141278
Rieder, C., Brachmann, C., Hofmann, B., Klein, J., Köhn, A., Ojdanic, D., Schumann, C., Weiler, F., Hahn, H.K.: Interactive visualization of neuroanatomical data for a hands-on multimedia exhibit. In: Proceedings of VMLS, pp. 37–41. Eurographics Association, Goslar (2013). doi:10.2312/PE.VMLS.VMLS2013.037-041
Röttger, D., Merhof, D., Müller, S.: The BundleExplorer: A focus and context rendering framework for complex fiber distributions. In: Proceedings of VCBM, pp. 1–8. Eurographics Association, Goslar (2012). doi:10.2312/VCBM/VCBM12/001-008
Schott, M., Martin, T., Grosset, A.V.P., Brownlee, C., Höllt, T., Brown, B.P., Smith, S.T., Hansen, C.D.: Combined surface and volumetric occlusion shading. In: Proceedings of PacificVis, pp. 169–176. IEEE, Los Alamitos (2012). doi:10.1109/PacificVis.2012.6183588
Schott, M., Martin, T., Grosset, A.V., Smith, S., Hansen, C.D.: Ambient occlusion effects for combined volumes and tubular geometry. IEEE Trans. Vis. Comput. Graphics 19(6), 913–926 (2013). doi:10.1109/TVCG.2012.306
Schultz, T., Sauber, N., Anwander, A., Theisel, H., Seidel, H.P.: Virtual Klingler dissection: Putting fibers into context. In: Proceedings of EuroVis, pp. 1063–1070. Eurographics Association, Goslar (2008). doi:10.1111/j.1467-8659.2008.01243.x
Shafii, S., Obermaier, H., Kolář, V., Hlawitschka, M., Garth, C., Hamann, B., Joy, K.I.: Illustrative rendering of vortex cores. In: Proceedings of EuroVis, pp. 61–65. Eurographics Association, Goslar (2013). doi:10.2312/PE.EuroVisShort.EuroVisShort2013.061-065
Svetachov, P., Everts, M.H., Isenberg, T.: DTI in context: Illustrating brain fiber tracts in situ. Comput. Graphics Forum 29(3), 1024–1032 (2010). doi:10.1111/j.1467-8659.2009.01692.x
Tarini, M., Cignoni, P., Montani, C.: Ambient occlusion and edge cueing for enhancing real time molecular visualization. IEEE Trans. Vis. Comput. Graphics 12(5), 1237–1244 (2006). doi:10.1109/TVCG.2006.115
Viola, I., Gröller, M.E., Hadwiger, M., Bühler, K., Preim, B., Sousa, M.C., Ebert, D., Stredney, D.: Illustrative visualization. In: IEEE Visualization Tutorials, Chap. 4 IEEE Computer Society, Los Alamitos (2005)
Wenger, A., Keefe, D.F., Zhang, S., Laidlaw, D.H.: Interactive volume rendering of thin thread structures within multivalued scientific data sets. IEEE Trans. Vis. Comput. Graphics 10(6), 664–672 (2004). doi:10.1109/TVCG.2004.46
Wood, J., Isenberg, P., Isenberg, T., Dykes, J., Boukhelifa, N., Slingsby, A.: Sketchy rendering for information visualization. IEEE Trans. Vis. Comput. Graphics 18(12), 2749–2758 (2012). doi:10.1109/TVCG.2012.262
Zhang, S., Demiralp, Ç., Laidlaw, D.H.: Visualizing diffusion tensor MR images using streamtubes and streamsurfaces. IEEE Trans. Vis. Comput. Graphics 9(4), 454–462 (2003). doi:10.1109/TVCG.2003.1260740
Zhang, S., Laidlaw, D.H., Kindlmann, G.: Diffusion tensor MRI visualization. In: Hansen, C.D., Johnson, C.R. (eds.) The Visualization Handbook, Chap. 16, pp. 327–340. Elsevier, Oxford (2004). doi:10.1016/B978-012387582-2/50017-4
Zhukov, L., Barr, A.H.: Heart-muscle fiber reconstruction from diffusion tensor MRI. In: Proceedings of Visualization, pp. 597–602. IEEE Computer Society, Los Alamitos (2003). doi:10.1109/VISUAL.2003.1250425
Zhukov, L., Barr, A.H.: Oriented tensor reconstruction. In: Hansen, C.D., Johnson, C.R. (eds.) The Visualization Handbook, Chap. 15, pp. 313–326. Elsevier, Oxford (2004). doi:10.1016/B978-012387582-2/50017-4
Zhukov, S., Iones, A., Kronin, G.: An ambient light illumination model. In: Rendering Techniques, pp. 45–55. Springer, Vienna (1998). doi:10.1007/978-3-7091-6453-2_5
Zöckler, M., Stalling, D., Hege, H.C.: Interactive visualization of 3D-vector fields using illuminated stream lines. In: Proceedings of Visualization, pp. 107–113. IEEE Computer Society, Los Alamitos (1996). doi:10.1109/VISUAL.1996.567777
Acknowledgements
I would like to thank all of the people who provided example images, tried to find material from old sources, or referred me to others—sometimes on very short notice. In particular, I would like to thank Joachim Böttger, Silvia Born, Ralph Brecheisen, Jesús Díaz-García, Mathias Goldau, Mario Hlawitschka, Radu Jianu, Daniel F. Keefe, Mathias Schott, Thomas Schultz, Pere-Pau Vázquez, and Anna Vilanova. I also specifically wish to thank those authors who provided demo applications of their publications that I could use to create my own example visualizations. Finally, I would like to thank Kai Lawonn and Wesley Willet as well as the anonymous reviewers for their valuable comments on drafts of this survey.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Isenberg, T. (2015). A Survey of Illustrative Visualization Techniques for Diffusion-Weighted MRI Tractography. In: Hotz, I., Schultz, T. (eds) Visualization and Processing of Higher Order Descriptors for Multi-Valued Data. Mathematics and Visualization. Springer, Cham. https://doi.org/10.1007/978-3-319-15090-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-15090-1_12
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15089-5
Online ISBN: 978-3-319-15090-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)