Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Research throughout the past few decades has led to a considerable number of visualization techniques that can be used in any given context. For instance, when a designer wishes to visualize hierarchies, techniques such as treemaps, trees, or sunburst diagrams can be used; for networks, matrices and graphs can be used; for information flows, Sankey diagrams and decision trees can be used; for temporal changes, small multiples, streamgraphs, and spiral charts can be used; and so on. Research in the cognitive and learning sciences has consistently demonstrated that different representations (e.g. visualizations)Footnote 1 of the same data can influence cognition in significantly different ways [1, 31, 40]. While different representations can enhance cognitive performance by encouraging certain perceptual and cognitive operations, they can also elicit various biases in thinking and reasoning [22, 38, 40].

Representational biases manifest in two major ways: constraints—limits on what aspects of data can be expressed by a representation; and salience—how a representation facilitates processing of certain aspects of data, possibly at the expense of others [38]. Constraints arise due to the syntactical limitations of how graphical primitives are arranged in representational forms [31, 36], whereas salience arises from how easily information can be extracted from a representation. Such biases are not necessarily bad, as the value of constraints and salient features is context-dependent. For instance, when visualizing logic problems to support reasoning about sets, certain graphical constraints in Euler diagrams are beneficial, as intersecting circles can readily express underlying logical relationships [35]. When visualizing networks to support reasoning about paths, matrices are limited in that they cannot directly express paths along multiple nodes, yet network diagrams do not have such a limitation [28]. However, matrices can make missing relations highly salient due to the existence of empty cells that can be perceived easily. Network diagrams, on the other hand, make such information only partially salient. Thus, the value of a representational bias (i.e., whether it is good or bad) depends on the context in which it is used. However, representational biases typically encourage thinking in certain ways at the expense of others, which can lead to the development of inaccurate or incomplete mental models. One way to mitigate this issue is to use multiple representations, thus providing different perspectives and encouraging multiple ways of thinking.

To work effectively with multiple representations, designers and users must be fluent in the various representations that are relevant for any given data and context. Representational fluency refers to knowledge and skills that involve being able to understand, use, create, evaluate, and translate between various representations. If individuals have fluency with multiple representational forms, they can employ appropriate practices that help mitigate the effects of representational biases. For example, when working with social network data, users can translate between a node-link diagram and an adjacency matrix depending on whether they want to identify paths in the network or the absence of relationships between two people. Representational fluency is considered necessary for professional discourse and practice in a number of fields including chemistry, physics, mathematics and biology. In this chapter, I argue that representational fluency should also be considered necessary for professional competence in information visualization and can be achieved through systematic training and education, in both formal and informal contexts. Thus, promoting representational fluency is a general strategy requiring concerted efforts of educators, researchers and practitioners.

Need for general strategies—Previous work in visualization has proposed general strategies for mitigating cognitive biases [7, 23, 30] as well as strategies for dealing with particular biases [8, 10]. While strategies focusing on specific visualizations, contexts, or biases are certainly useful and necessary, there is also a need for more general strategies. Extant scholarship on cognitive biases suggests that tackling specific biases, without complimentary general strategies, is not a sufficient approach, as biases often have multiple determinants. As Larrick [22] notes, “there is unlikely to be a one-to-one mapping of causes to bias, or of bias to cure”. Thus, developing strategies for mitigating particular biases, while useful, does not constitute a sufficient research plan for dealing with cognitive biases in visualization. In this chapter, I propose that promoting representational fluency among visualization designers and users is one strategy that can help mitigate biases at a more general level. This strategy can complement techniques that are devised for dealing with specific biases, visualizations, or users.

2 Representational Fluency

Representational fluency has been studied in various STEM disciplines having a considerable interest in visualization— especially chemistry (e.g. [14, 20]), biology (e.g. [26, 39]), and physics (e.g. [15]). In these disciplines, many phenomena are not available for direct perception—e.g. molecules, atoms, proteins and forces. As a result, visual representations are essential for teaching, learning, communicating, and conducting research [13]. Interestingly, although visual representations are indispensable for working with abstract data, similar attention has not been paid to representational fluency and its attendant concepts in information visualization.

Studies show that experts are more fluent than novices with multiple representations in their disciplines [6, 18]. In fact, the degree to which individuals exhibit representational fluency is strongly correlated with their level of expertise. Although this has not been investigated in information visualization, presumably both expert visualization designers and users should have higher degrees of fluency than novices.

Extant scholarship on representational fluency does not point to universal agreement on the characteristics of fluency, nor does it reveal a coherent theoretical underpinning. Various scholars refer to fluency in different ways, sometimes treating it as synonymous with representational competence. While there is no well-established conceptual framework for discussing fluency, there is a strong consensus on some of its key features. For instance, most scholars appear to agree on the following requirements for fluency—being able to make sense of the meaning of representations; being able to translate between equivalent or complementary representations; being able to devise new representations that are contextually appropriate; being able to evaluate and critique existing representations; and understanding the functions of various representations and how and when they should be used [15, 25].

Hill et al. [15] recently reviewed the literature on representational fluency and suggest that contributions have been made from three related perspectives—(1) representational competence, (2) meta-representational competence, and (3) meta-visualization. Each of these perspectives is elaborated below. While there is considerable overlap among these perspectives, it is useful to understand their individual origins and contexts, to see how they may provide value for information visualization.

Representational Competence—Representational competence typically refers to the ability to comprehend and use a set of domain-specific representations. Representational competence comprises the ability to properly extract information from a representation—i.e., to understand its syntax and semantics. Individuals may have representational competence if they can ‘see beyond’ the surface-level characteristics of representations to their common underlying features, and are able to translate between different representations of the same data [21].

Meta-representational Competence—While representational competence refers to skills with a certain set of representations, meta-representational competence transcends this view, focusing on an approach where individuals understand the rationale for using particular representations and the design strategies used to create them [9, 15]. ‘Meta’ here is not used in a self-referential fashion; rather, it is used in the spirit of the original Greek meaning of “beyond” or “after”—e.g. as in metaphysics. Thus, meta-representational competence can be thought of as beyond simply competence with representations. Meta-representational competence is evidenced by skills such as critiquing visualizations to assess their suitability in particular contexts, inventing new visualizations and describing why and how a visualization works in a particular context.

Meta-visualization—Here, visualization refers to the process of making meaning from external representations. In this view, visualization is more of a cognitive phenomenon than an external artifact—visualization refers to not only an external representation, but to the internal representation (e.g. mental model) and the relationships between the two. This perspective has been promoted by Gilbert [12, 13] in science education and particularly in chemistry education. In this perspective, meta-visualization refers to “metacognition in respect of visualization” [12]. Gilbert argues that, just as there are generalized forms of spatial intelligence, memory, and thinking, there could similarly be generalized forms of meta-visualization. This perspective emphasizes the metacognitive processes and skills required to make meaning from external representations—e.g. the monitoring and control of what is being seen, what aspects should be retained, how they should be retained, and how they might be retrieved for later use. This perspective is different from the other two, as it very strongly focuses on the integration of external and internal representations, on cognitive processes such as mental modeling and mental simulation, and on the skills needed to have metacognitive proficiency in making meaning from external representations.

3 Implications for Visualization Research and Practice

The three perspectives described previously, reflect decades of work on representational fluency across various disciplines. These perspectives can provide a general framework from which to pursue and particularize representational fluency in information visualization. For instance, from the perspective of representational competence, representations for different types of data, users, domains and/or contexts could be compiled and characterized. To be representationally competent in one area requires an understanding of the syntax and semantics of the representations involved. To make meaning of a treemap visualization, for example, one must understand that shapes nested within each other communicate hierarchical levels; that the size of the shapes encodes a value; and, perhaps, that colors encode categorical features of the data. If these conventions are not understood, one cannot comprehend the treemap, and thus does not have competence with this particular representation. This could be extended to include a range of visualizations for hierarchical data. An individual should be able to look at an icicle plot, a sunburst diagram, a treemap and a node-link tree diagram and see beyond the surface level marks and encodings, being able to recognize common features in the underlying data. They would be able to identify the same kinship relations in the different representations—e.g. parent-child, sibling, ancestor and descendant relations. They would know that some representations encode parent-child relationships explicitly with lines, while others encode them implicitly using features such as position, overlap or containment. Furthermore, given a treemap, they would be able to decode it and express the same data using an icicle plot.

An individual with meta-representational competence should be able to critique a visualization, describing why it is or is not appropriate in a given context, and should be able to devise a new representation based on the data and users’ tasks. While representational competence refers to the what and how of representations—e.g. what do they represent and how is it done, meta-representational competence refers to the why of a representation—e.g. why it works the way it does and why it is appropriate or inappropriate for the data and context. Individuals who are meta-representationally competent should be comfortable answering these types of ‘why’ questions in addition to ‘how’ and ‘what’ questions—e.g. why is a heatmap or parallel coordinates plot appropriate in a given context, how can one be constructed from the other, and so on.

The meta-visualization perspective is perhaps the least straightforward of the three perspectives. This perspective requires a more holistic lens, examining the distributed cognitive system comprising both internal and external representations and processes. Furthermore, it requires examining the metacognitive skills that operate on those processes. From this perspective, individuals should be able to articulate what kind of knowledge they are acquiring while viewing and interacting with visualizations, how and why they are storing various aspects and views on the data in memory, how they are relating this new knowledge to existing knowledge, and how they might retrieve it for later use for problem solving or other activities. Although Gilbert [12] suggests that meta-visualization can be assessed through various verbal protocols (e.g. think-aloud) and interviews, no detailed assessment methods have been devised. Further research is needed to determine how meta-visualization could be assessed in the context of information visualization.

3.1 Developing Representational Fluency

The strategy being proposed here will not be very effective if implemented only in specific cases to deal with specific biases. Although individual designers and users can indeed develop representational fluency, which should help mitigate potential biases that may arise, the ideal solution is for representational fluency to be promoted systematically during visualization education, training and practice. This suggestion is not unattainable, as it as already an accepted expectation in other disciplines such as chemistry, physics, biology and mathematics. For instance, for professional chemists, representational fluency is an inseparable aspect of their expertise.

An important caveat here is that we cannot always expect users of visualizations to be experts. As information visualization becomes more prevalent in everyday contexts, more non-experts are exposed to visualization techniques on a regular basis. For instance, as data journalism grows in popularity, more online news sources integrate visualizations into their news stories, which are read by the general public. While theoretically possible to train the general public to develop representational fluency with common visualization techniques (after all, most students are taught how to read and use bar and line charts, scatterplots and other common techniques in school), it is not reasonably practicable in the near future.

A more reasonable expectation is that visualization designers develop a high degree of representational fluency during their training. As a result, designers could anticipate when various representational biases may manifest themselves, and integrate deliberate strategies into their visualizations to help mitigate the biases. For instance, consider a designer wanting to visualize temporal change. If they know that an animated chart may have a representational bias, in that it is limited to expressing data only at particular points in time, they may choose to use a small multiples technique instead, which does not share the same representational bias [3]. Alternatively, the designer may implement an option for users to interactively translate between the animated chart and the small multiples view (which also has cognitive benefits other than mitigating biases; see [32]). Because of the designer’s representational fluency, they implement this option deliberately, knowing that it can help mitigate biases. Furthermore, depending on the context, the visualization tool may even encourage users to translate between the representations at certain points in time. With ongoing advances in intelligent mixed-initiative systems, such a prospect may not be so unlikely in the near future.

It is worth noting here that in order to most effectively mitigate cognitive biases, representational fluency must complement established knowledge on perception, cognition, decision making, semiotics, interaction design, visual encodings and other relevant topics. Representational fluency is not a panacea for all problems related to cognitive biases in information visualization.

3.2 Effect on Cognitive Processing

Much of the theoretical basis of cognitive debiasing suggests that successful strategies encourage individuals to move from surface-level to deeper-level thinking [22]. This can be viewed as a shift from ‘System 1’ to ‘System 2’ thinking in the language of Kahneman [19], or from ‘experiential’ to ‘reflective’ modes of cognition in the language of Norman [27]. Whatever the language, the intention is to shift cognitive processing from the fast, intuitive, unconscious mode to the slow, reflective, conscious mode. This is somewhat at odds with typical goals espoused in the visualization literature—namely, to offload as much cognitive processing as possible onto the perceptual system and onto external artifacts (e.g. visualization tools and computational processing).

Although it is generally desirable to offload cognitive processing when working with visualizations, mitigating cognitive biases may be an area in which it is beneficial to place more burden of cognitive processing onto users. However, increasing cognitive burden must be done in a principled fashion, as not all cognitive burden is beneficial. For instance, trying to make sense of a network visualization that is extremely complex, with considerable occlusion of nodes and edges, will certainly increase cognitive burden—yet this increase is not beneficial and could be avoided with better design. However, after working with one visualization for a while, translating to an alternative visualization may lead to increased cognitive burden—yet, this increase can be beneficial, as it forces the user into a more reflective mode of cognitive processing in which critical questions may be asked of the underlying data. Another strategy is to design interactions to deliberately influence cognitive processing, increasing the cognitive burden where designers deem appropriate (see [29, 34]). Indeed, strategies for manipulating cognitive effort through interactive interface design have been studied in the context of educational and learning technologies for many decades now (e.g. [5, 17, 33]).

Evidence for the benefits of deeper cognitive processing in cognitive bias mitigation can be found in the literature on cognitive debiasing. For instance, research has shown that counter-explanation, having individuals devise alternative explanations to observations, can help mitigate known biases, such as the explanation bias [2] and the hindsight bias [4]. Studies suggest that counter-explanation tasks may be beneficial by disrupting individuals’ focal hypotheses and engendering more thorough and careful thinking about the phenomena under investigation [16]. Although representational fluency is not the same as devising alternative explanations, seeing multiple representations of the same data may effect the same cognitive processes responsible for disrupting focal hypotheses. Other known strategies for mitigating biases, such as reference class forecasting [11], also rely on engaging individuals in deeper cognitive processing to be successful. As the strategy of deliberately engaging users in deeper cognitive processing has not traditionally been an area of focus for the information visualization community, there is a need for a research agenda that outlines the main challenges to be overcome.

3.3 Preliminary Research Agenda

Based on the work above, five broad challenges are enumerated for a research agenda focusing on representational fluency. These five challenges are not intended to be entirely orthogonal or exhaustive. It is worth noting that these challenges are very general and could likely be broken down into more specific sub-challenges. However, at this point, they give structure to a wide range of challenges in this area and can help direct future research. Future work will likely identify more specific challenges and appropriate methodologies for dealing with them. Based on work in other disciplines concerned with representational fluency and interactive visualizations, along with existing research on cognitive bias mitigation, these five points set the stage for a more elaborate research agenda to unfold in the future.

  1. 1.

    Identify a core set of representations in which all visualization professionals should be competent. This is a difficult challenge, as there are currently many dozens of existing visualization techniques and new ones are continually being devised. Additionally, not all visualizations are appropriate in all contexts, and some visualizations are intended for very particular contexts. It may not be possible to identify a universally agreed-upon set of representations. However, without at least a rough set of common representations, it is difficult to promote and assess fluency in them. It may be the case that core sets of representations are identified for different contexts, users and data, and fluency in one or more sets can be promoted and assessed.

  2. 2.

    Identify pedagogical practices that promote representational fluency. Without concerted efforts on the part of visualization educators, it is unlikely that designers can develop fluency with various representations. Educators need to develop pedagogical strategies and practices for promoting representational competence, meta-representational competence, and meta-visualization. Although work has been done in other disciplines, it is not necessarily transferable to information visualization. Well-trained visualization designers should be able to understand, for example, the semantics of various encodings in different representations, their particular representational biases, how and why they were created and when they are most appropriate to be used. They should also understand which visualizations can complement each other, and when and how users should be able to translate between them.

  3. 3.

    Develop ways of assessing representational fluency. Without both formal and informal ways of assessing individuals’ representational fluency, pedagogical practices go only so far. There is a need for the development of formally administered methods of testing representational fluency, as well as means of self-assessing fluency. For example, surveys such as the one by Hill et al. [15] could be developed for common visualization techniques. Other strategies, such as protocol analysis and eye-tracking [37], could also be explored. Educators could devise standardized tests in which various aspects of representational fluency can be assessed. To emphasize the more designerly aspects of visualization practice, various design challenges could be given. Classroom practices that encourage critical reflection, such as design critiques, could be employed both formally and informally to assess the development of students’ representational fluency.

  4. 4.

    Investigate strategies for appropriately engendering deeper cognitive processing. As discussed above, research on cognitive debiasing consistently shows that effective interventions tend to shift individuals’ thinking from a surface, unconscious level to a deeper, conscious level. Various strategies for implementing this in a visualization context can be explored. For instance, the representations that are made available to users, and the sequences in which they are made available could be manipulated; various interactions could be made available or unavailable to users at different points in time to encourage different cognitive operations; even micro-level aspects of interactions can be manipulated to promote more reflective thinking (e.g. see [24]). To tackle this challenge, the visualization community could advantageously borrow strategies from the instructional design and learning technologies literature.

  5. 5.

    Test effects on cognitive biases in various experimental settings. Although promoting representational fluency is a general strategy, which should have effects across a range of biases, it is still important to test bias mitigation with specific biases and visualizations. Experiments could be devised where individuals that are known to have representational fluency in at least some subset of representations (as determined by assessments mentioned in challenge 3 above) are given visualizations with known representational biases, and are given the means to interactively translate between representations while performing tasks. Various strategies devised in response to challenge 4 above could also be tested, shedding light on both the strategies of designers and the effects on users.

4 Summary

The development of representational fluency by visualization designers and users is one strategy for mitigating cognitive biases when working with visualizations. As representational fluency is a well-established expectation for professionals in a number of disciplines, it is not unreasonable to have the same expectation in information visualization. Furthermore, representational fluency is a serious topic for research and scholarship in other disciplines, and should be too in information visualization. Establishing representational fluency among visualization professionals will require a concerted effort on the part of educators, researchers and practitioners, and will likely have multiple benefits beyond mitigating cognitive biases. For instance, representational fluency can lead to better communication among researchers and practitioners; better trained designers who know when and how to implement particular visualizations and interactions; and users who are more visualization literate, which can be of benefit across a wide range of data-driven activities.