What are virtual items, and are they real?

Abstract

A central debate in the philosophy of virtual reality (VR) focuses on the reality of virtual items. Broadly, there are two main disagreements. Some views accept a metaphysical orientation to VR, and disagree on the reality of virtual items. For instance, David Chalmers (Disputatio 9(46):309-352, 2017, Disputatio 11(55):453-486, 2019, 2022) defends digitalism, the view that virtual items are real digital items. Neil McDonnell & Nathan Wildman (Disputatio 11(55):371-397, 2019), by contrast, defend fictionalism, which maintains that virtual items are unreal fictions. Other views, like Grant Tavinor’s (2021), reject the metaphysical orientation, arguing that focusing on the reality of virtual items is a mistake. In what follows I evaluate these two disagreements. I argue that experiences of virtual items depend on episodes of picture perception. Recognizing this reveals that the current debates are either easily resolved in favor of one view, or are not disagreeing about the reality of virtual items, but about what virtual items are.

Virtual reality technology has reinvigorated older philosophical debates, and introduced new ones. One new debate focuses on the reality of virtual items,¹ the items we encounter in virtual reality (VR).² In Reality + (2022) and elsewhere,³ Chalmers defends digitalism, the view that virtual items are digital items, and therefore real. One line of opposition comes from fictionalists like McDonnell and Wildman (2019),⁴ who take virtual items to be fictional and therefore unreal. Another line of opposition comes from Tavinor (2021), who argues that the metaphysical orientation assumed by digitalism and fictionalism is a mistake.⁵

While these positions are presented as alternatives, in what follows I argue that these views are not clearly in disagreement. This is because it is not clear that different participants share a conception of virtual items. Beginning with an ostensive definition grounded in the experience of virtual items, I argue that experiences of paradigmatic virtual items at least partly depend on picture perception.⁶ But given the complexities of picture perception, it is not clear what in the experience counts as the virtual item I offer four potential views of virtual items and evaluate the current literature through these. I argue that Tavinor’s anti-metaphysical view is driven by one conception of virtual items, and that Chalmers’ digitalism and McDonnell and Wildman’s (2019) fictionalism both fail to offer a single view of what virtual items are. Sorting through the possible options reveals that different views are largely speaking past one another. I conclude by outlining a path forward.

The argument proceeds in four sections. Section 1 offers an ostensive account of virtual items, arguing that virtual items are experienced in episodes of picture perception. Section 2 takes a closer look at picture perception to offer four views of virtual items. Section 3 evaluates the current debates, arguing that different positions are not clearly disagreeing about the reality of virtual items because they do not clearly share a conception of virtual items. Section 4 concludes the discussion.

1 Virtual reality and picture perception

It is not intuitively clear what virtual items are. On the one hand virtual technologies are relatively new. On the other hand ‘virtual’ has been interpreted in a number of ways, and it is not clear which of these, if any, is the right interpretation for VR.⁷ To avoid these issues, I propose beginning with an ostensive definition focusing on the experience of virtual items.

Virtual items are encountered when using a VR device. A simple VR device is the Google Cardboard (henceforth, Cardboard), a shaped cardboard embedded with lenses and a magnet, and which can house a modern phone. The setup and its use are illustrated below:

Using the Cardboard results in an impressive VR experience. The Cardboard’s VR meets all Chalmers’ criteria for full-blown VR: it is computer-generated because it depends on the phone’s computations. It is interactive because the phone’s rotation sensors, and magnet, allow one to look around the virtual world and interact with it respectively. And it is immersive because the experience elicits a sense of being present in a virtual world.⁸

The Cardboard make it apparent that virtual items are encountered by perceiving a particular type of picture. Removing the phone from the cardboard reveals this. VR applications display two side-by-side pictures. One has a VR experience when they see the two pictures’ contents stereoscopically fused into a single image. So plausibly, paradigmatic virtual items are encountered by seeing a particular type of picture.

The connection between virtual items and pictures is somewhat obscured in Chalmers’ and McDonnell & Wildman’s (henceforth MW) accounts, but is apparent in other accounts, like Grau’s Virtual Art (2003) and Tavinor’s (2021) The Aesthetics of Virtual Reality. It is also assumed in other fields, like psychology and human–computer interaction.⁹ Still, one may hesitate to accept the connection for at least two reasons.

First, the Cardboard’s pictures are atypical in ways that can seem to threaten their pictorial status. Under some conditions, pictures can become something other than a picture. For instance, Wollheim (1987) has argued that Trompe L’oeils are not pictures because they do not elicit the two-fold experience characteristic of pictures.¹⁰ In addition, a picture of X can be transformed into X through a series of changes. For instance, a picture of a door can gradually reproduce more of a door’s essential features to become a door. It can be painted realistically, on a piece of wood the size and shape of a door, with hinges to add interactivity.

Though I agree that a picture can cease to be one with sufficient alteration, VR pictures are not so elaborate. But see Ali (2023) for a broader discussion of what VR can reproduce and represent. The Cardboard’s pictures present contents that are slightly distorted at their periphery, moving, digital, and stereoscopic. But these features do not undermine the pictorial status of what we see because digital, moving, and stereoscopic pictures are still pictures, and pictures can have distorted contents for many reasons.¹¹

A more serious concern is that VR depends on more than just pictures. The Cardboard’s pictures are typically accompanied by sounds from the phone’s speaker, and are interactive. The pictured door may be purely visual, but the virtual door is visible, audible, and interactive.

While it is true that some virtual items are not purely visual, and some may not be visual at all,¹² this alone does not undermine the connection between virtual items and pictures.¹³ What this does show is that virtual items may not fully depend on pictures, in at least some cases.¹⁴ A virtual door may e.g. partly depend on emitted sounds, and partly on the pictured door. Moreover, some virtual doors are only visual. Despite its multimodal aspirations, current VR technology is primarily visual, with the primary addition being the head-mounted display (HMD).¹⁵ Even without interaction and auditory cues, HMDs can generate what we call a ‘virtual reality’ alone. So, at least paradigmatically, virtual items depend on perceived pictures.¹⁶

2 A closer look at pictures

Paradigmatic virtual items are experienced in episodes of picture perception. But if so, what part of the experience is the virtual item? To answer this, we need a closer look at what we see in pictures.

A picture is an ordinary object with a distinctive surface, and it is the distinctive surface that makes the object a picture. We can distinguish at least three components of pictures: the picture surface, the pictorial item, and the object a picture is of.¹⁷ Consider first the surface. In a traditional painting, this is the surface of a canvas painted with oils. Its equivalent in a VR device is the surface of the screen with lit pixels. Next there are pictorial items. What makes the surface of a picture distinctive is the way its visible properties are organized. Under some viewing conditions, this visual organization (e.g. of strokes on a canvas) results in an experience of pictorial items. In a picture of a cat, the pictorial cat appears in a particular style, and from a specific angle. Pictorial items are somewhat mysterious. On some views they are nothing over and above the picture surface, in others they are more ontologically independent.¹⁸ Depending on one’s view, pictorial items might be directly seen, or e.g. imagined. Finally, there is the item a picture is of. A painter can see or imagine a cat, and paint it. The cat painted with e.g. oils is the pictorial cat. But this is distinct from the cat the picture is of. The cat, unlike the pictorial cat, need not be anywhere near the surface. And it may not exist, if it is only imagined. Experiencing this item is mediated by the perception of the picture.

Distinguishing these components gives us a better view of the relationship between pictures and virtual items. In a VR device we encounter virtual items by seeing pictures. This means that virtual items are either some component of pictures, or something enabled by the experience of pictures.¹⁹ This gives us four potential accounts of virtual items. One possibility is that virtual items are (partly)²⁰ identical or constituted by picture surfaces. Another possibility is that virtual items are pictorial items.²¹ A third possibility is that virtual items are what the VR pictures are of. A final possibility is that virtual items are not some component of pictures, but something accessed through the picture.²²

While the above possibilities do not exhaust the available options for what a virtual item is, they do show that explicitly recognizing the role of picture perception in experiences of paradigmatic virtual items reveals a number of distinct construals of virtual items. Keeping these alternatives in mind allows us to better frame the literature’s disagreements.

3 The reality of virtual items

The debate on the reality of virtual items has two main disagreements. The first is about the metaphysical orientation towards VR. The second is a disagreement between virtual realists and irrealists, as represented by digitalism and fictionalism. We can consider each in turn.

Tavinor (2021) argues that the metaphysical orientation is a mistake.²³ It focuses on speculative, perfected VR that greatly abstracts from actual VR, and leads to confusions about VR’s nature and metaphysical import, while also leaving out relevant questions about VR as a new technological medium.²⁴ Tavinor defends these ideas after presenting his view, which also connects virtual items to pictures.²⁵ He maintains that virtualization lies between reproduction and representation, where “the structure and function are retained, while the customary material instantiation is lost.” (p.29). VR virtualizes the subject’s experience and interaction with a world by instantiating relevant features in a non-customary way. A central part occurs through VR’s distinctive pictures. Tavinor writes:

The objects I have argued to be perceived when using virtual reality media include the picture surface, the apparent 3D objects configured in this surface [i.e. the pictorial object], and the intentional object of the configurations [i.e. what the picture is of] … p.142 (bracketed statements mine)

Tavinor’s view of VR pictures is broadly in line with the view I have been presenting. But Tavinor also maintains that when using a VR device, we experience all three components of pictures. In place of virtual items, he argues that VR offers experiences of a myriad of objects types, since VR pictures can be of different item types e.g. some VR experiences are of fictional dragons, others of real cities. On this view, virtual realism and irrealism are not metaphysical theses, but distinct uses of VR.²⁶ Tavinor thinks this deflates the ontological debate between realists and irrealists, and offers a more parsimonious view that requires no additional (virtual) reality beyond what appears in pictures.

Although I am largely in agreement with and sympathetic to Tavinor’s account, I am less convinced that it undercuts the ontological debate. Even if VR is a picturing medium, there are ontological questions about pictures,²⁷ and these take on a new metaphysical significance with the possibility of experiencing VR with pictures. It is also not clear that Chalmers and MW share Tavinor’s view, that VR (primarily) offers an experience of what a picture is of.

Consider Chalmers’ digitalism. Digitalism does not seem to fit the pictorial framework neatly. But this is because the connection is obscured in Chalmers’ perception argument for digitalism:

• (1) When using virtual reality, we perceive (only) virtual objects.

• (2) The objects we perceive are the causal basis of our perceptual experiences.

• (3) When using virtual reality, the causal bases of our perceptual experiences are digital objects.

  –

• (4) Virtual objects are digital objects.²⁸

Chalmers’ first premise suggests a view different from Tavinor’s, who maintains that multiple things are perceived in VR. But the premise is also surprising for anyone with experience in VR, since we clearly see the picture surfaces we are facing.²⁹ Arguably, if we only see one thing, it is the screen’s surface. If Chalmers’ accepted this, and (at least partly) identified virtual items with surfaces, then realism would be easily attainable, since surfaces are real. But Chalmers rejects this view (see below).

Chalmers’ second premise is also unhelpful. It is not always clear what the causal basis of a perceptual experience is, or how this is related to what we consciously perceive. With the Cardboard, one can plausibly maintain that the surface is the perceived cause of the experience. Even if the surface is not the complete causal basis, because data structures govern the surface’s visual transformations which we also see, this at most suggests that we see both surface and digital items. This would also give us realism easily, since surfaces and data structures are real.³⁰

The problem is that Chalmers also rejects this view, maintaining that only digital items are seen. It’s worth noting the oddity of this view. Compare VR experience as Chalmers envisions it to seeing a sheet of metal that changes colors when magnets are placed behind it. Even if we think the occluded magnets are seen because they impact the sheet’s colors, it would be strange to maintain that only the magnets are seen. It is similarly strange to think digital items, but not surfaces, are seen in VR.

But Chalmers (2017) offers two cases to support his view. He compares his view to seeing the same actor on multiple TVs, and to photographic transparency.³¹ This suggests that virtual items are comparable to what a picture is of, not picture surfaces (or digital items). We ‘see’ the same actor on multiple TVs because the TV pictures are all of this actor. Similarly, Walton’s (1984) view about photographic transparency maintains that photographs (unlike other pictures) allow us to see what they are of because they have the right causal connection to items they are of. If this is Chalmers’ view, then he is in disagreement with Tavinor. The problem is that this makes digitalism clearly false, because as Tavinor points out, although some VR experiences are of real digital items, others are neither real, nor digital. In Google Earth VR we encounter many real but non-digital locations, and some VR is of fictions.

So, Chalmers’ view is ambiguous. Either virtual items are constituted by picture surfaces and the digital items governing their transformations, and realism is easily won, or virtual items are what VR pictures are of, and digitalism is wrong for the reasons Tavinor highlights.

Next, consider fictionalism. MW adopt a Waltonian view of fictions, where representational artworks offer real props for games of make-believe, which guide and determine imagined worlds. For Walton’s view, see Walton 1990. Applying this to VR, they write

proper engagement with VR is a kind of make-believe, featuring VR specific props and principles of generation. These VR specific props include digital elements like the particular images, sounds, and haptic feedback mechanisms employed by VR systems. Such elements really exist—we see, hear, and feel them whenever we engage with VR applications […] p.22

MW clearly reject the idea that virtual items are picture surfaces, or digital items, since these are only (real) props that support the (virtual) fictions. If we accept the first interpretation of Chalmers’ view, then MW and Chalmers’ disagreement is on what virtual items are, not their reality. Both agree that surfaces and digital items are real, but only Chalmers equates these with virtual items.

But although MW are clear on what virtual items are not, they are less clear on what they are, because it is not clear where fiction enters the picture. One possibility is that the surface is a prop for what the picture is of. For instance, the virtual door I see is e.g. the door to Bilbo Baggins’ house. If this is MW’s view, and we accept the second interpretation of Chalmers’ view, then the two views are in disagreement, and in disagreement with Tavinor. The problem is that this means MW are mistaken for the reasons Tavinor (and Chalmers (2019)) highlights: VR is not always of fictions.

A more charitable interpretation is that MW hold a Waltonian view of pictorial items. On this view, the surface of lit pixels is a prop, but it is a prop supporting a (fictional) pictorial item. The pictorial item is fictional, not the item the picture is of. This saves MW’s view from Tavinor and Chalmers criticism, because even VR documentaries depend on (fictionally) taking lit pixels to be pictorial items. But this view is also not in disagreement with Chalmers or Tavinor, since neither focuses on pictorial items. Moreover, even if Chalmers’ view is taken to be about pictorial items, this would save the disagreement, but not in a way that helps resolve the ontological question. This is because neither Chalmers nor MW offer substantive theories of pictorial items, or draw on the relevant literature or cases.

So, if virtual items are some component of pictures, then either there are easily resolvable disagreements, or else participants are disagreeing about what virtual items are, not about whether or not they are real. This leaves one possibility. In Section 2 I argued that virtual items may be something accessed via pictures, rather than a picture component. On this view, VR pictures are like portals,³² or transparent glass that offers a window to a world beyond the glass (a world distinct from the picture surface, pictorial items, and what the picture is of). This view construes the debates as follows: Chalmers and MW disagree on the nature of the world beyond pictures, and Tavinor disagrees with the metaphysical extravagance of postulating such a world.

While I think this view is worth investigating, I think there is currently little reason to take it seriously. First, seeing pictures is unlike seeing through a window. Unlike pictures, windows are transparent and can be seen through. Pictures, including VR pictures, by contrast have opaque surfaces that cannot be seen through. At most, we can see something in the picture (a pictorial world), or ‘see’ what the picture is of. Second, even if we accept that pictures have this sui generis transparency, it is unclear how pictures manifest this portal-like feature. For instance, are VR pictures distinctive in enabling this? If so, why? If not, then what other cases have this transworld significance? Without addressing these issues, it is difficult to say much more about the view.

4 The way forward

I argued that current debates about the reality of virtual items largely speak past one another because different views offer distinct conceptions of virtual items. To make headway in the debate, we first need a common account of what virtual items are. And given the connection between virtual items and pictures, this task is connected to accounts of pictures and picture perception.

The connection to pictures simplifies the debate in some ways. For instance, pictorial items are sometimes described as ‘virtual’,³³ which suggests that we should further investigate the relationship between them and virtual items, since they may shed light on one another. But the connection also complicates some issues, like skeptical VR scenarios. If our world is virtual, and the virtual is related to the pictorial, then our (virtual) world at least partly depends on (hyper-immersive) pictures. Though I agree with Chalmers that our position is not as bad as the skeptic thinks, the connection to pictures shows that it is also not as good as Chalmers thinks. This is because we are both ignorant of what our world is made of (atoms or bits), and of whether or not it is of something.³⁴ The evil demon may have designed our reality as a twisted image of Eden, or simply fabricated it. But we would not know that our (virtual) reality is of Eden, or an imagined world, because we would not know whether our world is of anything to begin with.³⁵