Visual Information and Scientific Understanding

Abstract

Without doubt, there is a widespread usage of visualisations in science. However, what exactly the epistemic status of these visual representations in science may be remains an open question. In the following, I will argue that at least some scientific visualisations are indispensible for our cognitive processes. My thesis will be that, with regard to the activity of learning, visual representations are of relevance in the sense of contributing to the aim of scientific understanding. Taking into account that understanding can be regarded as an epistemic desideratum in its own right, I will argue that, at least in some instances, no understanding can be achieved without the aid of visualisations. Consequently, they are of crucial importance in this process. Moreover, to support this thesis we will make use of some findings in educational psychology.

1 Outline: Philosophy’s Problem with Visual Information

Without doubt, there is a widespread usage of visualisations,¹ such as photographs, graphs, diagrams, drawings, etc., in science. However, there is still no consensus amongst philosophers of science with regard to the epistemic status of such visual representations. Obviously, there are at least three possible approaches with regard to the question of their status. Firstly, we could deny that visualisations play any epistemic role whatsoever. Explaining their integration in scientific communication would then amount to the thesis that they are mere decorations, added for psychological purposes only such as attracting attention (see Carney and Levin 2002). Secondly, we could take a more moderate stance and admit that visual representations in science serve important heuristic means. Integrating them into communicative acts allows complex data to be arranged in a comprehensible way, highlighting the essentials and presenting all the relevant details at first glance (see Kulvicki 2010). Thirdly, we could defend the more controversial thesis that (at least some) visual representations are indispensable in scientific publications and presentations as they can make accessible certain information which cannot be transmitted otherwise (see e.g. Elkins 2010).

Interestingly, the difficulty in coping with the visual can be detected in a variety of different philosophical discussions. The following three examples are related to the philosophy of science and the philosophy of film. In this sense, they do not only highlight the fact that the above-mentioned difficulty is discussed with regard to different topics in philosophy, but also in different domains of this academic discipline.

Our first example belongs to the realm of philosophy of science. Here we are confronted with the difficulty to assess the epistemic status of visualisations, if we discuss the role of thought experiments in science. In this context, James Robert Brown has suggested the thesis that the latter should be regarded as particular tools of grasping the laws of nature by literally seeing them during the process of imagination that is inherent to thought experiments (see Brown 2011, ch. 5). Even a more general description of thought experiments, offered by him, includes this important visual element. “We visualize some situation; we carry out an operation; we see what happens, and we draw a conclusion” (Brown and Fehige 2011, Sect. 1). In contrast to this, John D. Norton is of the opinion that the information offered by such experiments of our mind can completely be stated in the form of ordinary verbal arguments, i.e. all information can be translated into linguistic expressions without loss. Hence, he thinks that the picturesque element involved in this imaginative act is not relevant to the function that thought experiments fulfil in science (see Norton 1996, p. 335ff.).²

Paisley Livingston brings up a similar point in the context of film philosophy. Here he argues that if films can be said to make genuine contributions³ to philosophical debates, then the transmitted information has to be paraphrased to make it part of the debate. But if this paraphrasing is possible the filmic element does not play a relevant part in constituting the potential philosophical contribution. Furthermore, if it is not possible to express the filmic content in verbal arguments, we will have to stop talking about it anyway—as it can never be part of a philosophical discussion in the proper sense (see Livingston 2006, p. 12f.).

Finally, we are confronted with a similar difficulty with regard to images in science—and, hence, we are back in the philosophy of science. Here Laura Perini makes us aware of the fact that the hesitation of many philosophers of science in appreciating the apparently significant role that visual representations play in scientific publications is a result of their conviction that all that matters in science are verbal arguments. Philosophers endorsing such a view defend the claim that images of whatever kind cannot be a proper part of those arguments. Visualisations belong to a completely different system of representation. Moreover, the elements of such systems cannot be truth-bearers, as traditionally only propositions are said to fulfil this role.⁴ Hence, those philosophers claim that visual representations cannot adopt the role of premises and conclusions in argumentation (see Perini 2005, p. 914).

In a nutshell then, in all of these cases we are confronted with a particular visual element—either as a part of thought experiments or films, or directly as printed visual representations. Some philosophers and doubtlessly many scientists take these as playing a significant role in knowledge-seeking enterprises such as rejecting a scientific hypothesis with the aid of a thought experiment. However, some philosophers claim that the apparent epistemic contribution of visualisations is only the result of a translation of their content into classical verbal arguments. And it is those arguments which serve the epistemic purpose, not the image itself. Thus, there is obviously a tension here between, on the one hand, the practical side of science and, on the other hand, the theoretical side of philosophy.

From this initial situation at least two questions can be derived. Firstly, what role, if any, does the information provided in a visual form play with regard to cognitive processes in science? Is it merely a redundant way of transmitting propositional knowledge, since we will have to translate it into verbal arguments anyway? Or does it serve a different epistemic function and, if so, does it maybe also offer some additional epistemic value? And, secondly, if we want to defend the latter thesis, then how exactly do visualisations fulfil their epistemic role?

Of course, visual representations serve a variety of different functions in science. Some may be of an epistemic kind and others may not (see Downes 2012, p. 117f.).⁵ In the following, I will select just one of these possible tasks correlated with the explanatory context where visualisations are used to communicate research results to others (see Frankel and DePace 2012, p. 13). More precisely, my considerations will be about the context of students’ education with the aid of textbooks. Taking the act of learning in communicative situations as our starting point has two obvious advantages for discussing the epistemic role and status of visual representations in science. Firstly, learning is commonly understood as an essentially cognitive task.⁶ In this sense, being successful at learning something normally implies two important epistemic desiderata: knowledge and understanding (see Kosso 2007, p. 175). Secondly, focusing on educational purposes in science also allows us to transfer some of our results to the context of genuine research, possibly assessed as the more relevant domain with regard to epistemological considerations. The point is that we can assume that even senior scientists are expected to learn something (more or less new) from taking a published research article into account. Hence, clarifying the role and status of visual representations with regard to the education of students may also tell us something about the contribution of such representations to cognitive processes in this context.

What then is the cognitive task of visualisations in educational contexts? Again, there are different possible answers. Here, I will concentrate on the contribution of visual representations to achieving scientific understanding. To clarify their particular role in this context, we will make use of some insights from the realm of educational psychology. Furthermore, my defence of the thesis that visualisations can make a substantial contribution in this realm is correlated with another assumption recently put forward in epistemology, namely regarding understanding as an epistemic desideratum with an intrinsic epistemic value (see Kvanvig 2003, p. 186). Thus, if we can show that at least some visual representations fulfil a significant role in facilitating scientific understanding, then it can indeed be argued that they are indispensible in science.

2 Scientific Education: Knowledge and Understanding

Let us start with a short example. Take a traditional physics textbook (see e.g. Tipler 1994; Tipler and Llewellyn 2012) and open it at random at a certain page. Inspecting the information offered in this context, e.g. about acoustics or thermodynamics, we find both written text and visual representations of different kinds. Now, what contributions are made by these different parts with regard to the act of learning?

Our initial thesis might be that the relevant parts are those that enable the student to acquire knowledge during her process of education. Following Gottlob Frege (1993, p. 30ff.), proponents of a more traditional philosophical approach claim that this epistemic achievement presupposes the propositional form of the information presented, as only propositions are said to be able to bear truth-values. Moreover, propositions are normally regarded as being of a sentence-like structure—that is, having a content which I could express as a that-clause. Obviously, visual representations are designed differently. Their content is not presented in a sentence-like structure.⁷ According to the view associated with Frege visual representations could only be said to transmit knowledge, if their assumed content can somehow be translated into verbal phrases.⁸ But if we have to do this, the question then arises of why using them at all? To agree with the suggestion of translation would lead us to a devaluation of the visual representation involved, similar to the theses of Norton and Livingstone with respect to the visual element in thought experiments or films.

I am not totally convinced of this classical account in philosophy, as I think that one can know much more than one can express verbally. Be this as it may, I have already mentioned that, apart from propositional knowledge, learning is normally also associated with the aim of understanding. Peter Kosso, for example, reminds us of the fact that solely memorising propositions is not what we expect our students to do—especially not in science (see Kosso 2007, p. 175). Usually, we do not want them to parrot hypotheses and statements during an exam. On the contrary, a scientific training ideally means making students part of the community of researchers, i.e. enabling them to apply acquired knowledge to new instances, to critically reflect upon this information, and possibly also to correct some of its parts. This is exactly the point where the aspect of understanding comes in.

Scientific understanding is commonly regarded as an ability to coherently fit new items into one’s knowledge system and to apply the newly acquired information to solve further tasks and puzzles. Wesley C. Salmon phrases this in the following way: “[…] we have scientific understanding of phenomena when we can fit them into the general scheme of things, that is, into the scientific world-picture” (Salmon 1993, p. 12f., his italics). But how exactly should we conceive of this fitting-relation? What does Salmon suggest when he claims that ‘to understand something’ means ‘being able to fit it into “the general scheme of things”’? An answer to this question is offered by Jonathan L. Kvanvig who also emphasises that this fitting-relation is the crucial difference between knowledge and understanding. He writes “[…] that understanding requires, and knowledge does not, an internal grasping or appreciation of how the various elements in a body of information are related to each other in terms of explanatory, logical, probabilistic, and other kinds of relations that coherentists have thought constitutive of justification” (Kvanvig 2003, p. 192f., my italics).

Moreover, Kvanvig thinks that understanding is a species of knowledge. That is, if a student understands that x, she also knows that x. In this sense, propositions play a role here too and understanding is regarded as being of an additionally epistemic value which is spelled out in the grasping of connections. In the context of education, Kosso puts this benefit in the following way: “Understanding reveals the larger landscape and includes the ability to apply one idea to other situations without being given detailed instructions” (Kosso 2007, p. 176). In this sense, understanding is an important goal of scientific education in the long run, enabling students to do their own research.

However, Kvanvig’s claim that understanding is a species of knowledge is somewhat problematic. It constitutes the starting point for the discussion whether understanding is factive or not.⁹ What does this mean? Obviously, we can have knowledge without understanding (see Kvanvig 2003, p. 191), as the above example of the student learning by rote shows. But is it also possible to claim that you can understand a state of affairs or a phenomenon without having at least true beliefs about it? In the philosophy of science this question becomes virulent with regard to idealisations used by scientists in their cognitive processes. Obviously, idealisations, such as most experiments yielding observational data or ceteris paribus laws, are often the starting point for cognitive processes in science, though they are, strictly speaking, not true. They are simplifications of actual phenomena or processes. Particular aspects are intentionally excluded in these cases and in this way the amount of information is reduced. Can we nonetheless have an understanding of the (actual) phenomenon or subject matter in question? Philosophers tend to discuss the factive status of understanding more or less exclusively with regard to this difficulty (see e.g. Elgin 2007; Mizrahi 2012).

An exception in this context might be Catherine Z. Elgin (1993, 1996). She reminds us of the fact that “[w]e also understand pictures, words, equations, and diagrams. Ordinarily these are not isolated accomplishments; they coalesce into an understanding of a subject, discipline, or field of study” (Elgin 1993, p. 14).¹⁰ Yet, if we accept the traditional philosophical framework connected with Frege, we have to admit that pictures or diagrams do not express propositions. Consequently, they can be neither true nor false (see ibid., 27). Connecting this with Elgin’s statement, we are forced to say that the starting point of the cognitive process of understanding can also be constituted by instances which are not truth-bearers at all—at least not according to traditional philosophical accounts. Thus, the question about factivity takes another twist with regard to visual representations. We do not start from false propositions here, but from entities which can be neither true nor false.

Despite this difficulty, the thesis that visualisations play an important role in scientific understanding can nonetheless be supported by different analyses of educational psychologists (see e.g. Müller et al. 2012; Schnotz 2002; Vekiri 2002). Let us take a look at some of their results to find out more about the potential role of visual representations in facilitating scientific understanding.

3 Educational Psychology on the Role of Visualisations in Science Education

The way in which visualisations can be useful in educational contexts is comprehensively discussed by Ioanna Vekiri (2002). She presents three different theoretical approaches from the realm of educational psychology to explain the contributions of graphical displays¹¹ in students’ learning processes. With respect to the educational merits of these visualisations then, there are, on the one hand, those theories dealing with the positive effects on remembering information (see ibid., 262). Both dual coding and dual retention approaches belong to this set of theories. On the other hand, there are approaches that are subsumed under the heading of visual arguments dealing with the transmission and processing of information offered visually.

Although Vekiri focuses solely on graphical displays such as diagrams, the first functional aspect of visualisations—the one about memorising—also affects pictorial representations such as photographs (see Schnotz 2002, p. 107). Proponents of this first branch of theories state that there are different cognitive subsystems in the human brain to process the different kinds of representations. Whereas linguistic information is solely processed and encoded in the verbal cognitive subsystem, visual or pictorial information is processed and encoded both within the verbal and the imagery system (see ibid.). In this sense, images leave two different memory traces, so to speak, a verbal and a pictorial one. This dual way of information storage is then taken to explain why people can more easily remember visual representations than text alone. In this context, Vekiri highlights the fact that recent studies in neuropsychology and cognitive science have shown that these theoretical assumptions actually obtain (see Vekiri 2002, p. 267ff.).¹²

However, even though we might acknowledge the positive effect of visual representations on information storage and retrieval, this does not explain how they can enhance students’ understanding. Obviously, this special functional feature of visual representations does not make a difference on the level of information transmission—propositional or not—but only on the level of remembering information. Therefore, the more interesting account in our context is the one called “visual argument”. What is this about?

Right from the start we should point out that the label “visual argument” might be slightly misleading here. Theories of this approach are not concerned with arguments in the philosophical sense. Thus, we are not discussing the validity or structure of arguments, i.e. premises, conclusion, and inferential reasoning.¹³ However, this psychological approach focuses on the ability of visual representations to transmit information and to enable the recipient to grasp complex relations among them. “Visual argument concentrates on the perceptual and interpretation processes that take place when learners extract meaning from graphical representations. It claims that graphical displays are more effective than text for communicating complex content because processing displays can be less demanding than processing text” (Vekiri 2002, p. 262). In this sense, proponents of this account state that visualisations enhance the process of learning on the following levels.

Firstly, such representations offer information both about their individual elements and their relations (see ibid., 281). Secondly, graphical displays may allow us to perform comparative tasks simply by using perceptual faculties, i.e. without engaging in long interpretations (see ibid., 282).¹⁴ Just by looking at a bar graph the student may simply see the difference in length expressing a difference in quantity, etc. Finally, this perceptual accessibility of the data and its connections also allows a more economical handling of cognitive resources. The point is that for learning about the relevant relations the recipient does not have to search the whole text and store the different elements in her working memory which is prone to error. She can just take a look at the graphical display which organises the data in the relevant ways (see ibid., 282) and think about possible problem solutions in the meantime.

Of course, in order to obtain these benefits in the cognitive process of learning certain characteristics on the part of the recipient, the instructional design (see Schnotz 2002, p. 114ff.), and the design of visual representations (see Vekiri 2002, p. 301ff.) are presupposed. Especially the first aspect has to be highlighted. “Visuo-spatial text adjuncts and other forms of visual displays can support communication, thinking, and learning only if they interact appropriately with the individual’s cognitive system. Accordingly, the effects of visuo-spatial adjunct aids depend on prior knowledge, cognitive abilities, and learning skills” (Schnotz 2002, p. 113). Visual representations may indeed transmit even complex information by mere perception, i.e. by merely looking at them. That visualisations can be put to work in this way, however, presupposes a trained eye, so to speak. Thus, it would be wrong to claim that visual representations are understood automatically without prior learning (see Scholz 2009, p. 40ff.). Oliver R. Scholz, for example, shows that there are many different steps involved in our understanding of a single picture (see Scholz 1993, 2010). This is, however, no reason to devalue visualisations in comparison to verbal representations, as we are confronted with the same difficulty in this realm, too.

Moreover, after students have mastered this initial obstacle, visual representations can support the cognitive process of learning on at least three different levels: firstly, by showing the relation of individual information; secondly, by making information directly perceptually accessible; and, thirdly, by enabling a more efficient usage of cognitive resources. This last aspect is highlighted by Vekiri. “Also, displays support thinking during problem solving because they reduce the amount of information that must be maintained in working memory” (Vekiri 2002, p. 288).

As a concluding remark, let us now analyse how these three positive cognitive effects of visual representations might be correlated with the potential role that they play in scientific understanding.

4 Understanding: The Epistemic Merit of the “Picturesque”?

In a sense, it could be stated that visual representations are mere heuristic tools in the educational context, namely in the sense of supporting the cognitive process of learning. However, this also means that they enable students not only to acquire propositional knowledge but also to achieve an understanding of the information presented. Acknowledging the fact that understanding is an epistemic desideratum of its own now establishes the possibility of a particular twist in argumentation. Whereas pointing to the heuristic function of visualisations usually implies the devaluation of their epistemic status, we can defend the opposite point of view. If they can facilitate understanding, and understanding is independently epistemically worthwhile, then it can be stated that they make a substantial epistemic contribution.

But how do visualisations facilitate scientific understanding? Our discussion of the results from educational psychology suggested at least three different contributions of visual representations to the learning of scientific data. These contributions can now be related to the pursuit of scientific understanding in the following ways.

It was said that scientific understanding is about grasping connections between concepts, theories, and the like. This implies the comprehension of explanatory or probabilistic relations between newly acquired information and background knowledge, as Kosso claims. The obvious part to be played then by visual representations is to show these connections—that is, to visualise them literally. Tree diagrams might be a striking example in this context. Students are not only expected to learn something about particular items, but also about the relations in their field of study. Visualisations can highlight such relations in an immediate fashion and, thus, support the cognitive process of understanding.¹⁵

Furthermore, images might tell their recipients something about their individual visual elements, even though the students lack the relevant concepts. For instance, comparative tasks can be performed with the aid of visualisations when no concepts are available. Thomas E. Wartenberg points this out and discusses pictures in bird guides as an example. He thinks that these visual representations enable birdwatchers to identify the objects of interest during their flight, i.e. when observation conditions are not optimal. Furthermore, the observers do not need to know how to verbally describe the difference between possible candidates (see Wartenberg 2006, p. 26f.).¹⁶ Christopher Peacocke also highlights this special feature of visual representations. He states that, even though, recipients might lack the relevant concepts to describe what a picture shows we might nonetheless learn something about the particular visual appearances.¹⁷ That this is the case can be shown by pointing to our ability to recognize entities in their natural surroundings, although we learned about their visual appearance only by looking at images (see Peacocke 1987, p. 395). Thus, having learned what a buzzard looks like during its flight by depictions in a bird guide, the student will be able to recognize it in the wild the next time she observes one.

Moreover, recipients are able to directly access the information presented visually by using—more or less—their perceptual abilities alone. This is the second merit of visual representations in transmitting information acknowledged by educational psychology. At least some visualisations¹⁸ allow us to access their information directly with the aid of our primary human sense—by visual perception. Of course, written language is equally perceived by our visual senses, thus it is not only the mode of perception that is relevant here. What is of importance, however, is that the perception of at least some visual representations may allow us to grasp their meaning without interpreting them in a way that verbal language presupposes. According to the results of educational psychology, as pointed out above, there is empirical evidence supporting the thesis that after an initial mastering of how to understand the visual content of different representations, the latter can become directly accessible to the students.

Furthermore, this perceptual mode of access also enables recipients to make use of correlated skills that have developed in the course of evolution. In this sense, Zachary C. Irving (2011) argues for a fundamental role of visual representations concerning scientific understanding. He discusses the difference between visual and numerical representations and highlights the fact that because of the limitations of human cognitive capacities the former are particularly useful for the understanding of large data sets. His primary example is about scatter plots which, according to Irving, are especially useful for detecting patterns among the data (see ibid., 780f.). As an example we can take a look at the “Hertzsprung-Russell Diagram” (see e.g. http://chandra.harvard.edu/edu/formal/variable_stars/bg_info.html), showing the correlation between temperature and magnitude of stars. Obviously, by simply looking at the plot we can see that most of the stars in our surrounding belong to the main sequence.

I would agree with Irving that some visual representations are especially worthwhile as they enable pattern detection among data. However, I would relate this merit to our abilities as visual observers rather than to our cognitive limitations. That human observers are particularly suited to performing this task is, for example, suggested by a web-based project in astronomy, called “Galaxy Zoo” (http://www.galaxyzoo.org/). Recent sky surveys have yielded huge amounts of data that now has to be analysed. “Galaxy Zoo” is part of this analytical project. Here volunteers—also laymen without special training—participate in classifying galaxies photographed by space telescopes such as Hubble according to their shapes. Obviously, the project organisers rely on their volunteers’ ability of pattern detection. Contrary to Irving’s example, however, this task cannot be performed on the level of numerical data, i.e. as computational operations by IT devices. The point is that there is a certain vagueness involved in classifying these objects and this vagueness cannot be removed by sticking to numerical data, but can only be reduced by consistent results of different human classifiers. That humans are particularly skilled in the task of pattern detection is undoubtedly a consequence of evolutionary processes. Thus, making information available in a way that also activates these skills can enhance our understanding by connecting the cognitive processing of information to these abilities.

The last point then is about the more economical handling of cognitive resources, if information is presented in the visual format. What educational psychology here suggests is that visualisations can constitute a kind of relief for our cognitive system. In this sense, images might be regarded as a kind of extended memory system, so to speak, though I do not want to relate this to the debate about the extended mind here.¹⁹ However, they keep information and relations among the data available while we think about problem solutions. We do not have to store all the information in our working memory during this process. In this sense, visualisations might provide the necessary cognitive resources to work out the relevant connections to fully understand a particular topic.

Hence, all of these aspects of information transmission with the aid of visualisations can enhance scientific understanding—also without being reduced to propositions. In this sense, I would agree with James Robert Brown (see e.g. Brown and Fehige 2011, Sect. 3.2) and others that there is more to the picturesque, the visual format, than can be translated into verbal arguments. At least in some instances visual representations are inescapable in science.

5 Conclusion

In a nutshell, our above analysis started with the assumption that visualisations obviously play a crucial role in science. However, the question that we had to consider was what exactly the functions of these visual representations are. In this context, I tried to defend the thesis that visual representations can make important contributions in epistemic processes and that some are even indispensible in these contexts. Supporting evidence was then put forward in our discussion of some results from educational psychology. In this context, it was shown that visualisations are indeed often of crucial importance with regard to the aim of scientific understanding. Furthermore, taking into account the fact that understanding can be epistemologically regarded as an epistemic desideratum in its own right and combining this with the result that in some instances scientific understanding can only be achieved via visualisations, this can be regarded as speaking in favour of the suggested thesis of an (at least partial) indispensability of the visual in scientific cognitive processes.