Keywords

Introduction

Despite demonstrations of perceptual impairments in schizophrenia as far back as Kraepelin (1903), and although the visual system is the most understood area of cognitive neuroscience, there has been much less work focused on vision in schizophrenia compared to other domains (e.g., memory, executive function) (Silverstein & Keane, 2011b). This, despite an ever increasing body of literature indicating that visual impairments are prevalent among individuals with schizophrenia, and that they are important in terms of understanding the nature of the condition and its course. For example, approximately 25–30 % of individuals with schizophrenia report visual hallucinations (Waters et al., 2014), and the rate of patients reporting visual distortions (in the domains of brightness, motion, form, and color) is at least double that (Cutting & Dunne, 1986; Phillipson & Harris, 1985). Laboratory measures of these domains of visual processing have long track records in experimental psychology, and have provided many demonstrations of impairments in schizophrenia research (reviewed below). Importantly, visual abnormalities are clinically significant. For example, visual distortions are associated with suicidal ideation even after controlling for factors such as psychotic symptoms and auditory distortions (Grano et al., 2015). Laboratory indices of visual impairments are related to impaired cognition (Calderone, Hoptman, et al., 2013; Haenschel et al., 2007), and social cognition (Butler et al., 2009; Green, Hellemann, Horan, Lee, & Wynn, 2012; Kim et al., 2010; Kim, Shim, Song, Im, & Lee, 2015; Laprevote, Oliva, Delerue, Thomas, & Boucart, 2010; Lee, Gosselin, Wynn, & Green, 2011; McBain, Norton, & Chen, 2010; Norton, McBain, Holt, Ongur, & Chen, 2009; Silverstein et al., 2010, 2014; Turetsky et al., 2007; Vakhrusheva et al., 2014), poor reading ability (Martinez, Revheim, et al., 2012), lower overall functioning (Green et al., 2012; Rassovsky, Horan, Lee, Sergi, & Green, 2011), and poorer treatment response (Silverstein, Keane, et al., 2013; Silverstein, Schenkel, Valone, & Nuernberger, 1998). In several cases, visual changes appear to be specific to schizophrenia, in that they are not found in patients with other psychotic or nonpsychotic psychiatric disorders (Uhlhaas & Silverstein, 2005a, 2005b). These visual changes are not limited to patients with an established illness, however. They are also, in some cases, found in children, adolescents, and young adults at high-risk for schizophrenia (Hebert et al., 2010; Koethe et al., 2009; Mittal, Gupta, Keane, & Silverstein, 2015; Revheim et al., 2014; Schubert, Henriksson, & McNeil, 2005). One study demonstrated that visual distortions were more sensitive to later conversion to psychosis than abnormalities in other domains, including auditory distortions or thought disorder (Klosterkotter, Hellmich, Steinmeyer, & Schultze-Lutter, 2001). Another demonstrated that problems with visual acuity and other aspects of visual and ocular functioning in childhood predicted later development of schizophrenia better than other sensory impairments, in high-risk and general-population samples [(Schubert et al., 2005), see also Schiffman et al. (2006)]. In short, visual disturbances represent symptoms, endophenotypes, biomarkers, and predictors for schizophrenia.

In addition, visual cortex, and activity therein, can serve as a useful model of broader aspects of coordinated brain function and its impairment (Phillips & Silverstein, 2003). Laboratory tasks that emphasize small-scale aspects of neural integration (e.g., tasks of visual gain control) (Huang, Hess, & Dakin, 2006), as well as tasks that involve long-range integration [e.g., frontal–parietal connectivity as it is involved in contour integration (Castellano, Plochl, Vicente, & Pipa, 2014; Dima et al., 2009)], can be useful in demonstrating the integrity of small- and large-scale networks, respectively, in schizophrenia. This is important because issues such as disruptions in network function, excitation, and inhibition may be more relevant to understanding schizophrenia than performance on any single laboratory task developed in prior decades. There has, arguably, been a paradigm shift since the 1980s from focusing on individual cognitive processes (and the earliest point in the processing sequence where abnormality exists) (Cromwell, 1984), to searching for the common foundations for the multiplicity of cognitive problems in schizophrenia, and the types of network (rather than regional) failures that cause them [e.g., Adams, Stephan, Brown, Frith, & Friston, 2013; Carr & Wale, 1986; Clark, 2013; Cohen & Servan-Schreiber, 1992, 1993; Corlett, Frith, & Fletcher, 2009; Corlett, Honey, Krystal, & Fletcher, 2011; Phillips & Silverstein, 2003]. This theme will be developed further in the section on contextual modulation below.

Altered Subjective Visual Experience in Schizophrenia

A question that comes up often when the topic of vision in schizophrenia is raised is: “why study vision when most patients don’t have visual hallucinations, or do things like bump into walls?Footnote 1” Although it is true that the frequency of visual hallucinations is lower than that of auditory hallucinations, a much higher proportion of people with schizophrenia experience visual distortions (Bunney et al., 1999; Cutting & Dunne, 1986). These experiences were noted by seminal thinkers such as Jung, who described “ isolation symptoms ” including perceptions of walls bending and bulging (Jung, 1958). Consider the following two examples of disturbances in perceptual organization: “Everything I see is split up. It’s like a photograph that’s torn in bits and put together again. If somebody moves or speaks, everything I see disappears quickly and I have to put it together again”; and “I have to put things together in my head. If I look at my watch I see the watch, watchstrap, face, hands and so on, then I have got to put them together to get it into one piece” (Chapman, 1966). Or, the following recollection by a therapist: “I asked him what he had in mind. He told me that he frequently saw the shape of things change before his eyes and that he often felt that he saw colorful objects sail through his field of vision” (Lenzenweger, 2011). Although it is rare that patients are asked about such experiences during clinical assessment, or in research studies, a group of German investigators incorporated a subscale of items involving visual disturbances into a research interview designed to assess “basic symptoms” of schizophrenia (Ebel, Gross, Klosterkotter, & Huber, 1989; Huber & Gross, 1989)—the Bonn Scale for the Assessment of Basic Symptoms (BSABS). Scores on this subscale had a higher sensitivity for predicting conversion to psychosis in a high-risk sample than any other symptom category (Klosterkotter et al., 2001). The categories of vision disturbances on the BSABS, and examples of each, are depicted in Table 1.

Table 1 Visual disturbance items from the BSABS, and examples of each type

The examples above in Table 1 highlight the wide variety of visual disturbances experienced by people with schizophrenia. They are experienced during the prodromal stage, at first episode, and later in the illness, although they are most pronounced in untreated patients (Kelemen, Kiss, Benedek, & Keri, 2013; Phillipson & Harris, 1985). They may also be significant in terms of other symptom formation. For example, Conrad (1958) and Matussek (1952, 1953, 1987) both discussed how fragmentation, changes in form, and hyper-intensity of visual stimuli can lead to changes in the sense of self, and to delusional mood or the feeling that the world is changing, being drained of meaning, taking on new meanings, or headed for an apocalyptic event [for reviews of this literature, see Uhlhaas and Silverstein (2005b) and Uhlhaas and Mishara (2007)]. They can also lead to other odd beliefs, generated to try to explain the origin of the altered perceptions (Chapman, 1966). To date, however, the influence of visual changes on symptom formation has not been studied empirically.

The onset of new visual symptoms may also be revealing, as it may signal developing eye disease. For example, I recently evaluated a patient who reported visual hallucinations of grid-like figures (tessellopsia ), and geometric shapes when under stress. These types of images are common in cases of eye disease (and migrane), and are rarely associated with psychiatric syndromes, but can be caused by certain street drugs (Ffytche, 2007), use of which was ruled out in this patient’s case. Optical coherence tomography (a form of retinal imaging) of this patient revealed severe retinal nerve fiber layer thinning in both eyes, and significant thinning in multiple macular regions of both eyes, suggesting that this symptom was due to retinal degeneration (and did not represent an increase in psychosis). Note that it is important to consider that even when visual symptoms have a retinal origin, they still may be interpreted or elaborated with delusional thinking. A revealing case demonstrating this involved retinal detachment and subsequent visual disturbances that were incorporated into preexisting somatic and paranoid delusions in a person with schizophrenia. Due to the history of delusions, an ophthalmologic consult was delayed. However, it eventually revealed a retinal detachment (Brda & Tang, 2011), which is a reversible condition (in most cases, when surgery is not delayed by too long).

In short, given the long history of reports of visual disturbances in the clinical schizophrenia literature, and their clinical significance, it would appear that greater study of these phenomena is warranted. Although there has been relatively little research on phenomenological changes in vision in schizophrenia, there does exist a wealth of experimental laboratory data on visual changes. In the next section, I review laboratory findings from several well-researched paradigms. Whereas at this point, with a few exceptions (Kelemen et al., 2013; Keri, Kiss, Kelemen, Benedek, & Janka, 2005), it is not known whether subjective visual disturbances are correlated with laboratory demonstrations of visual impairments in schizophrenia, such a connection has already been demonstrated in the autism literature (Davis, Bockbrader, Murphy, Hetrick, & O’Donnell, 2006).

Laboratory Psychophysical Studies of Visual Processing in Schizophrenia

By “visual processing” I mean the mental activity that generates our visual experiences, consistent with Firestone & Scholl (in press).

The majority of the evidence on visual processing impairments in schizophrenia comes from studies within experimental psychopathology. As with all areas of schizophrenia research involving performance-based measures (e.g., attention, memory, learning, executive functioning, processing speed), studies have shown impairments in nearly all domains of visual processing. However, from the many studies that have been done, several consistent themes have emerged. These involve findings regarding contrast sensitivity, spatial frequency processing,Footnote 2 backward masking, motion detection, perceptual organization, and effects of prior knowledge on interpretation of visual input (including size constancy and other visual illusions).

Contrast Sensitivity. Contrast sensitivity refers to the ability to detect changes in luminance (i.e., darker to lighter regions or vice versa) (see Fig. 1). It is of interest because it is a low-level visual process, whose impairment would suggest basic visual system disturbance in schizophrenia (i.e., it would be difficult to explain in terms of reduced cognitive control). The results of multiple behavioral and electrophysiological studies indicate that contrast sensitivity is altered in schizophrenia (Butler et al., 2005, 2009, 2012; Butler, Silverstein, & Dakin, 2008; Cadenhead, Dobkins, McGovern, & Shafer, 2013; Calderone, Martinez, et al., 2013; Keri, Antal, Szekeres, Benedek, & Janka, 2002; Keri, Kelemen, Benedek, & Janka, 2004; O’Donnell et al., 2006; Slaghuis, 1998). Studies of medicated patients have consistently demonstrated reduced contrast sensitivity. However, the role of dopamine-receptor blocking medication in causing this effect needs to be ruled out, as data from animal modeling (Bodis-Wollner, 1990) and from healthy humans (Bulens, Meerwaldt, van der Wildt, & Keemink, 1989) show that administration of antipsychotic medication decreases contrast sensitivity. Further, research with schizophrenia patients indicates that decreases in contrast sensitivity are medication dose-dependent (Keri et al., 2002). The type of medication may matter as well, with one study finding that patients on first-generation antipsychotic medications demonstrated poorer contrast sensitivity compared to both healthy controls and patients on second-generation medications (Chen, Levy, et al., 2003). Adding further complexity, Harris, Calvert, Leendertz, and Phillipson (1990) found that (first-generation) antipsychotic medication decreased contrast sensitivity for medium and high spatial frequencies, but increased it at low spatial frequencies (note—the latter result has not been found in other studies). And, the combination of antipsychotic and antidepressant medication [a combination frequently prescribed due to high rates of depression in schizophrenia (Siris, 2000)] appears to be especially detrimental to contrast sensitivity (Sheremata & Chen, 2004). An open question is the role of factors other than medication in causing these effects in chronic patients. It is known that contrast sensitivity for medium to high spatial frequencies declines with age (and that this is due to retinal and neural effects, not due to changes in the lens) (Elliott, 1987) and that striatal dopamine-receptor availability reduces with age (Backman, Nyberg, Lindenberger, Li, & Farde, 2006). What is not clear though is whether these changes are accelerated in patients with schizophrenia. Preliminary published data on the latter issue have been negative (Nakajima et al., 2015). However, in an ongoing study in my group, with subjects within the age range of 18–65, Brian Keane has observed that in schizophrenia patients (n = 67), contrast sensitivity (averaged across all spatial frequencies) declined with age (r = .33, p = .006), with identical findings for the subgroup of patients with 20/20 or better vision (n = 50). In contrast, the relationship was not significant for control subjects (n = 50, r = .12, p = .42). These data suggest a potential interaction between schizophrenia and/or dopamine-receptor blocking medication and age.

Fig. 1
figure 1

Contrast sensitivity can be measured in many ways. In Calderone, Martinez, et al., (2013), subjects were presented with either (a) low spatial frequency condition, or (b) a high spatial frequency condition. The task was to determine on which side (left or right) the grating appeared. Contrast (the difference between lighter and darker portions of the grating) was adjusted trial-to-trial using a staircase procedure (Calderone, Martinez, et al., 2013). In this figure, lower contrast stimuli are at the front, and higher contrast stimuli are at the back, in examples (a, b). Reprinted from NeuroImage , Volume 67, Calderone et al., Comparison of psychophysical, electrophysiological, and fMRI assessment of visual contrast responses in patients with schizophrenia, 2013, p. 155, with permission from Elsevier

Adding to the controversy are data from studies of unmedicated schizophrenia patients, and high-risk patients, who have demonstrated either relatively normal contrast sensitivity (Cadenhead et al., 2013), or increased contrast sensitivity compared to healthy controls (Chen, Levy, et al., 2003; Keri & Benedek, 2007; Kiss, Fabian, Benedek, & Keri, 2010). The difference between medicated and unmedicated patientsFootnote 3 would appear to be due to excesses of retinal and brain dopamine during acute psychotic episodes (Brandies & Yehuda, 2008), and reduction in dopamine levels following administration of dopamine-receptor blocking psychiatric medications [reviewed in Silverstein and Rosen (2015)]. This statement is based in part on evidence from single-photon emission computed tomography (SPECT) and positron emission tomography (PET) studies that support the traditionally held view that the acute psychotic phase of schizophrenia (especially in younger patients) is characterized by striatal hyperdopaminergia , whereas striatal dopamine approaches normal levels in periods of symptom remission (Kegeles et al., 2010; Laruelle, Abi-Dargham, Gil, Kegeles, & Innis, 1999), and may be significantly lower than normal in chronically ill patients (Elkashef et al., 2000). However, not all the data on medication effects on contrast sensitivity in schizophrenia suggest that the picture is this simple [reviewed in Skottun and Skoyles (2007)]. Adding to the complexity is that different brain regions appear to be characterized by excessive (e.g., the striatum) or reduced (e.g., prefrontal cortex, midbrain) dopamine at the same time during acute psychosis (Slifstein et al., 2015), different dopamine-receptor types can have different effects on brain function (Tost, Alam, & Meyer-Lindenberg, 2010), and D2 receptors can exist in high or low affinity states (Seeman et al., 2006). Nevertheless, much additional evidence indicates that dopamine activity affects contrast sensitivity. This includes: (1) the similarity between reduced contrast sensitivity in Parkinson’s disease (a condition characterized by loss of dopaminergic neurons) [reviewed in Silverstein and Rosen (2015)], and findings in medicated schizophrenia patients: (2) the beneficial effects of the dopamine precursor L-DOPA on contrast sensitivity (Gottlob & Stangler-Zuschrott, 1990); (3) the relationship between dopamine and gain controlFootnote 4 (Cohen & Servan-Schreiber, 1993); and (4) more general findings of hyper-excitability in cortical networks in schizophrenia at first episode with decreases over time (Anticevic et al., 2015; Silverstein, All, et al., 2012). An important question for future research involves the relative contributions of retinal vs. brain dopamine, since dopamine manipulation is known to affect retinal functioning (Bodis-Wollner & Tzelepi, 1998; Brandies & Yehuda, 2008; Silverstein & Rosen, 2015; Witkovsky, 2004). It will also be important to clarify effects of the disorder on dopamine in the occipital lobe, a topic that has been largely unstudied. Finally, it should also be noted that antipsychotic medications, in addition to reducing available dopamine, can cause toxic maculopathy (Lee & Fern, 2004) and increase the risk for cataracts (McCarty et al., 1999), which can also affect contrast sensitivity as well as performance on any visual function test.

Other than the role of dopamine, the biological basis of excesses and reductions in contrast sensitivity in schizophrenia has been hotly debated. A prominent theory is that reduced contrast sensitivity at lower spatial frequencies reflects magnocellular pathway dysfunction (Butler et al., 2008). The evidence for this (but not for impaired contrast sensitivity) has been sharply criticized on several grounds (Skottun & Skoyles, 2007), including that the stimuli used in a number of studies would not be expected to preferentially activate the magnocellular pathway, and that the studies did not discriminate between subcortical magnocellular pathway effects and cortical dorsal stream effects (the visual pathway that receives much magnocellular input).Footnote 5 More recently, contrast sensitivity deficits in schizophrenia have been viewed in terms of a reduction in the general mechanism of gain control (Butler et al., 2008, 2012). More will be said about this in the section on contextual modulation below.

Spatial Frequency Processing. Spatial frequency refers to the number of changes in luminance (i.e., light–dark–light changes) within a unit of space (typically 1° of visual angle) (see Fig. 2). Spatial frequency processing is important because a current view of primary visual cortex (V1) holds that it contains detectors tuned to specific spatial frequencies, whose function is to decompose visual images into component parts (Everson et al., 1998; Issa, Trepel, & Stryker, 2000). Assessment of processing over a range of spatial frequencies is typically done within the context of contrast sensitivity testing, but some studies have manipulated spatial frequency content during face or object perception and examined subsequent effects on recognition and decision-making. As with contrast sensitivity, most studies demonstrate impairments in patients, but these differ as a function of chronicity, medication status, and patient symptoms.

Fig. 2
figure 2

Examples of low, medium, and high spatial frequency grating pairs. In O’Donnell et al. (2002) subjects were required to discriminate whether the right and left gratings in each pair were the same or different in spatial frequency. Reprinted from Journal of Abnormal Psychology , O’Donnell et al., Spatial frequency discrimination in schizophrenia, 2002, with permission from the American Psychological Association

Many studies demonstrate reduced low spatial frequency processing in schizophrenia (e.g., Calderone, Hoptman, et al., 2013; Kiss, Janka, Benedek, & Keri, 2006; Martinez, Hillyard, et al., 2012; O’Donnell et al., 2002). Impairment at medium spatial frequencies has also been demonstrated (O’Donnell et al., 2002). Here again though, the role of medications needs to be considered. For example, whereas increased LSF processing was observed in untreated first episode patients, with normalization of LSF processing after treatment (Kelemen et al., 2013), in chronic patients, an increase in LSF processing and decrease in medium and high spatial frequency processing after treatment were reported (Harris et al., 1990). This apparent discrepancy is most likely related to different types of dopaminergic changes in first episode and chronic patients, as noted above. Consistent with the hypothesis that chronicity affects spatial frequency processing, a recent study observed that whereas patients within 10 years of their first psychotic episode demonstrated impairments in the processing of only low spatial frequencies, patients who had been ill for over 10 years demonstrated decreases in contrast sensitivity at all spatial frequencies (Shoshina & Shelepin Iu, 2013). An important issue here is whether impaired encoding of spatial frequency information really worsens over a wider range of spatial frequencies with illness chronicity, or whether this effect is due to other factors, such as reduction in visual acuity in schizophrenia. The latter is observed in the premorbid state (Schiffman et al., 2006; Schubert et al., 2005), and in diagnosed patients (Viertio et al., 2007), and differences between patients and controls are sometimes not observed on spatial frequency processing tasks when the groups are equated on visual acuity (Keane, Erlikhman, Kastner, Paterno, & Silverstein, 2014; Silverstein, Keane, et al., 2014). Also, in an ongoing study in my laboratory, Brian Keane has demonstrated that when groups are matched on visual acuity, patients (n = 50) and controls (n = 50) show only a trend toward a between-group difference at the higher end of the low spatial frequency range, but no difference at high spatial frequencies. Note that matching groups on visual acuity are not the same as merely ensuring that both groups have “normal or corrected to normal vision,” as is typically done. This is because even small differences within the normal range can have significant effects on performance on measures of visual perception (Keane, Kastner, Paterno, & Silverstein, 2015).

Although reduced sensitivity to low spatial frequencies has been interpreted to indicate a magnocellular pathway dysfunction , this has been challenged on several grounds, including evidence that processing of higher spatial frequencies (which would be primarily processed by the parvocellular system) is also often affected, and that some effects may be medication related (Skottun & Skoyles, 2007) (but see footnote 6 for rebuttals to this argument). Given the close relationship between spatial frequency processing and contrast sensitivity, it is possible that impairments in both in schizophrenia reflect reduced gain control (Butler et al., 2008) or a more general reduction in stimulus responsivity or gain (Skottun & Skoyles, 2013). In the latter view, the apparent widening of the range of spatial frequency processing impairment with increasing chronicity may reflect an overall worsening of sensitivity to stimuli in general, perhaps associated with age- (Backman et al., 2006) and medication-related (Howes & Kapur, 2009) reductions in brain dopaminergic activity (D2 binding) and age-related death of retinal dopaminergic neurons (Witkovsky, 2004). Conversely, what appears to be increased sensitivity to low spatial frequency stimuli in untreated first episode patients may reflect increased striatal dopamine levels and cortical excitability early in the illness.

Masking. Masking refers to the phenomenon whereby a briefly appearing target becomes harder to see when it is preceded or succeeded by a distracting visual stimulus (see Fig. 3). In a typical masking paradigm, both stimuli are presented briefly (e.g., 50–300 ms), and the time between stimulus onsets (stimulus-onset asynchrony, SOA) or the time between the offset of the first and the onset of the second stimulus (interstimulus interval, ISI ) are manipulated to determine the amount of temporal separation necessary for adequate target identification. Impairments in masking have been demonstrated in schizophrenia in forward (where the mask precedes the target) and backward masking, using multiple variations on the basic masking paradigm (see Fig. 3). In general, patients require longer SOAs or ISIs in most masking conditions.

Fig. 3
figure 3

Examples of targets, masks, and response functions from different masking paradigms. For these tasks, participants were asked to identify the location of the gap in one side of target square. In the first two types of masking, the target could appear in one of four possible locations on the screen. In the middle column, paracontrast refers to the condition where the mask surrounds but does not integrate with the target, in forward masking; metacontrast refers to use of the same stimuli in a backward masking paradigm. In the case of 4-dot masking, the mask specified which square in an array of four squares was the target. Reprinted from Schizophrenia Bulletin , Volume 37(4), Green et al., Visual masking in schizophrenia: Overview and theoretical implications, 2011, p. 702, with permission from Oxford University Press

Early masking studies interpreted the need for longer SOAs or ISIs in schizophrenia as being due to slowness of processing (Saccuzzo, Hirt, & Spencer, 1974; Saccuzzo & Schubert, 1981), and therefore (depending on the masking paradigm) to excessive integration of target and mask, or interruption of target processing by the mask. However, later work demonstrated that the processing demands of the mask (including the level of meaning it contained) affected SOAs and ISIs, and therefore suggested that the issue was not slowness per se, but rather, problems in the perceptual organization of stimuli and the integration of perceptual and conceptual information in a post-sensory stage known as short-term visual memory (Knight, 1984; Knight, Elliott, & Freedman, 1985; Knight & Silverstein, 1998, 2001; Rabinowicz, Opler, Owen, & Knight, 1996; Weiss, Chapman, Strauss, & Gilmore, 1992). This is consistent with data indicating that backward masking problems in schizophrenia are due to more than problems in stimulus feature assembly alone, and with data indicating altered dynamic coupling between the lateral occipital complex (LOC) (a region involved in object processing) and more anterior regions, including those in the frontal lobe, during masking task performance in people with schizophrenia (Harvey et al., 2011).

Some studies have suggested that abnormal masking functions in schizophrenia may reflect magnocellular pathway impairment (Cadenhead, Serper, & Braff, 1998; Schechter, Butler, Silipo, Zemon, & Javitt, 2003). However, while some results may support this view, data from many studies do not [reviewed in Skottun and Skoyles (2009)]. Some of the discrepancy may be due to the use of different masking paradigms in different studies, since a structural equation modeling of masking data suggests that different paradigms require the use of different processing mechanisms (Rassovsky, Green, Nuechterlein, Breitmeyer, & Mintz, 2005). However, recent data suggest that a magnocellular account is probably too simple even where it does fit the data (Green, Lee, Wynn, & Mathis, 2011). An alternative explanation is that schizophrenia is characterized by overly broad tuning of visual cortex neurons , leading to imprecise, noisy, and unstable representations in LOC, and to subsequent delays in reentrant processing of visual information [note—outside of orientation tuning (Robol et al., 2013; Rokem et al., 2011; Schallmo, Sponheim, & Olman, 2013a), the types of visual features for which overly broad tuning may exist has yet to be specified]. The hypothesis that target representation formation is adequate for recognition (in the absence of a mask), but nevertheless excessively noisy or weakly registered is consistent with older data showing abnormal ERP activity during pre-mask target processing during a masking task in schizophrenia (Patterson, Spohn, & Hayes, 1987). Of note, masking deficits are most pronounced among schizophrenia patients with histories of poor premorbid functioning (Green et al., 2011; Knight, 1984; Knight & Silverstein, 1998).

Unlike data on contrast sensitivity and low spatial frequency processing in schizophrenia, masking findings cannot be explained in terms of levels of dopaminergic activity, since findings are relatively similar in unmedicated and medicated patients, and in high-risk populations and patients (Green et al., 2011), groups that would be expected to differ markedly in dopaminergic tone and cortical excitability (Nitsche, Monte-Silva, Kuo, & Paulus, 2010), based on data reviewed above. Data also indicate that while blunted neural responses in LOC are involved in the masking deficit in patients, this is not the case in unaffected relatives, suggesting contributions from a second factor in patients (Green et al., 2011). A candidate for this second factor is reduced suppression. One reason for this is that it can be difficult to distinguish broadened orientation tuning from surround suppression,Footnote 6 a form of contextually modulated gain control (Schallmo et al., 2013a). Therefore, the possibility that the neuronal tuning problems involved in masking deficits in schizophrenia may be secondary to a more basic, and illness-related, process that influences receptive field signaling (e.g., suppression or inhibition from surrounding neurons), must be considered. This is especially so given findings that GABA agonists enhance tuning in visual cortex (Leventhal, Wang, Pu, Zhou, & Ma, 2003), that reduced GABA level in the occipital lobe is related to reduced orientation-specific surround suppression in schizophrenia (Yoon et al., 2010), and that, in general, surround suppression effects in schizophrenia appear to be state-related (see below).

Perceptual Organization. Perceptual organization refers to the processes by which individual elements of sensory information are collectively structured into larger units of perceived objects and their interrelations (Palmer, 1999). Since 1961 (Snyder, Rosenthal, & Taylor, 1961), over 50 studies have demonstrated reduced visual perceptual organization or impaired Gestalt perception in schizophrenia across various paradigms, labs, and countries [reviewed in Silverstein and Keane (2011a) and Uhlhaas and Silverstein (2005a)]. Among psychiatric conditions, this dysfunction appears to be specific to schizophrenia, as it has not been observed in other disorders.Footnote 7 A general theme throughout this literature is that schizophrenia patients experience perceptual organization difficulties when stimuli are composed of spatially noncontiguous elements (i.e., object contour is fragmented) and/or are novel, while their basic processing of closed shapes and edges, and of symmetry, is intact (Chey & Holzman, 1997; Knight, Manoach, Elliott, & Hershenson, 2000; Silverstein, Bakshi, Chapman, & Nowlis, 1998; Silverstein, Bakshi, Nuernberger, Carpinello, & Wilkniss, 2005). Research indicates that perceptual organization dysfunction is related to the illness per se rather than to antipsychotic treatment; for example, there is no relation between either oral dose or blood level of medication and performance on perceptual organization tasks (Knight, 1992; Silverstein et al., 2009, 2010; Spencer et al., 2004), and impairments have been demonstrated in unmedicated patients (Frith, Stevens, Johnstone, Owens, & Crow, 1983; Keri, Kiss, et al., 2005). As with backward masking, multiple studies demonstrate that among schizophrenia patients, perceptual organization dysfunction is closely associated with poor premorbid adjustment [see Knight and Silverstein (1998) and Uhlhaas and Silverstein (2005a) for reviews].

Since the late 1990s, my lab and others have investigated perceptual organization in schizophrenia using variants of the contour integration (CI) paradigm , which originated in the basic vision literature (Braun, 1999; Chandna, Pennefather, Kovacs, & Norcia, 2001; Field, Hayes, & Hess, 1993; Kovacs, 2000; Kovacs & Julesz, 1993). CI involves representing continuous boundaries and shapes on the basis of the relative positions and orientations of spatially discrete edge elements. It is typically measured as the ability to detect or make a judgment about the shape, position, or presence of a closed contour made up of noncontiguous elements, embedded within a display of randomly oriented elements (see Fig. 4). CI tasks typically manipulate perceptual organization by either: (1) changing the number of noise elements relative to contour elements (i.e., signal–noise ratio) (Field et al., 1993; Kovacs, Polat, Pennefather, Chandna, & Norcia, 2000; Silverstein, Kovacs, Corry, & Valone, 2000); (2) adding randomly applied amounts of orientational jitter to the contour elements, thereby reducing the correlation between the orientations of adjacent elements and weakening smoothness of the contour line or curve (Silverstein et al., 2009); or (3) increasing the spacing between contour elements (Keane et al., 2012; Li, Piech, & Gilbert, 2006), thereby requiring greater top-down, prefrontal, input to perceptual processes (Ciaramelli, Leo, Del Viva, Burr, & Ladavas, 2007). To date, tests relying on the first two of these manipulations (see below), but not the third (Keane et al., 2012), have shown perceptual organization impairments in schizophrenia. In the latter case though, a wider range of contour element spacing needs to be studied to reach a firm conclusion about prefrontal contributions to CI in schizophrenia.

Fig. 4
figure 4

Examples of stimuli from the JOVI (CI) task: (a) left-pointing contour from no-background catch trial; (b) right-pointing contour from outline catch trial; (c) left-pointing contour from 0° jitter condition; (d) left-pointing contour from 11° jitter condition. Note that these images emphasize the center portion of the stimulus containing the contour. Actual test stimuli included a larger background area. Reprinted from Neuropsychologia, Volume 75, Silverstein et al., Cortical contributions to impaired contour integration in schizophrenia, 2015, p. 470, with permission from Elsevier

The most robust CI task to emerge from this literature is the Jittered-Orientation Contour Integration task (JOVI), which uses the orientational jitter manipulation. With the JOVI and its precursor tasks, studies have shown that people with schizophrenia are less able to detect and make shape judgments about closed but noncontinuous contours when compared to various healthy and psychiatric control groups (Butler et al., 2013; Feigenson, Keane, Roche, & Silverstein, 2014; Keane et al., 2012, 2014; Kozma-Weibe et al., 2006; Schallmo et al., 2013a; Schallmo, Sponheim, & Olman, 2013b; Schenkel, Spaulding, DiLillo, & Silverstein, 2005; Schenkel, Spaulding, & Silverstein, 2005; Silverstein et al., 2000, 2006, 2009; Silverstein, Keane, et al., 2012; Uhlhaas, Phillips, Schenkel, & Silverstein, 2006; Uhlhaas, Phillips, & Silverstein, 2005). Past CI studies in schizophrenia have also demonstrated that, while performance does not vary from the acute to stabilization phases of illness in briefly hospitalized (i.e., ~2 weeks) patients (Feigenson et al., 2014), it becomes worse with longer illness chronicity and a lower level of functioning (Keane, Paterno, & Silverstein, Submitted; Schenkel, Spaulding & Silverstein, 2005; Silverstein et al., 2006; Uhlhaas et al., 2005).

Exploration of the brain bases of CI in schizophrenia has been limited to one ERP study (Butler et al., 2013) and two fMRI studies (Silverstein et al., 2009, 2015). Butler et al. (2013) and Silverstein et al. (2009) both found reduced processing, compared to healthy controls, in visual regions known to subserve CI, based on prior studies from healthy humans and monkeys (Altmann, Bulthoff, & Kourtzi, 2003; Kourtzi, Tolias, Altmann, Augath, & Logothetis, 2003; Volberg & Greenlee, 2014) (e.g., V2, V3, V4, LOC). These findings pointed to disturbances in the coordination of feedforward (bottom-up) stimulus assembly processes with reentrant top-down disambiguation (from higher occipital, and temporal, areas) to increase the salience of contours and to inhibit background noise—an iterative integration mechanism proposed to subserve CI in healthy observers (Chen et al., 2014). In addition to disturbances in the occipital lobe, however, Silverstein et al. (2009) found reduced prefrontal cortex and parietal lobe activation in patients. These findings fit with recent data indicating that in addition to visual cortex, frontal–parietal connectivity and other mechanisms supporting higher-level top-down control are involved in CI and object processing (Castellano et al., 2014; Hanslmayr, Volberg, Wimber, Dalal, & Greenlee, 2013; Li, Piech, & Gilbert, 2008; Sun et al., 2012; Volberg, Wutz, & Greenlee, 2013). Data from the initial fMRI and ERP studies of CI are supported by findings from studies of perceptual closure, where subjects view fragmented line drawings of familiar objects. These studies, which utilized visual evoked potentials and other later ERP waveforms, indicated both impaired early (Foxe, Doniger, & Javitt, 2001) and late (Doniger, Foxe, Murray, Higgins, & Javitt, 2002) contributions to perceptual organization impairment.

An ERP marker of the moment of holistic perception of an object with interrupted contour is a negative deflection in the ERP, occurring at approximately 270 ms after stimulus onset, and known as closure negativity (Ncl) (Doniger et al., 2000). Studies indicate that Ncl amplitude is attenuated in schizophrenia patients (compared to controls) and this is associated with reduced activation within occipito-temporal and parietal–occipital regions (Doniger et al., 2000, 2002), as well as reduced activity in prefrontal areas (Sehatpour et al., 2010; Sehatpour, Molholm, Javitt, & Foxe, 2006). An fMRI study of perceptual closure confirmed reduced activity within these regions in schizophrenia (Sehatpour et al., 2010). Reduced Ncl was also found in the Butler et al. (2013) CI study discussed above.

In a recently published fMRI study (Silverstein et al., 2015), we again observed reduced CI in schizophrenia, but this was associated with increased activation in patients compared to controls in LOC and in parietal regions involved in visual attention. The groups did not differ in activity in visual regions posterior to LOC (e.g., V1–V4), and there were no regions where controls demonstrated greater activation than patients. We interpreted these findings as reflecting adaptation and perceptual learning effects in controls, due to a presumed greater ability than patients to benefit from a lengthy prior testing session outside of the scanner, and to a higher proportion of perceivable contours in this task than in the task versions used in prior studies (which used higher levels of orientational jitter). This view is supported by much data indicating that, in healthy people, repeated exposure to stimuli can lead to activation decreases as processing becomes more efficient [e.g., Yotsumoto, Watanabe, and Sasaki (2008)]. It appears then that in schizophrenia, even after repeated exposure to the stimuli, the ability to rapidly represent the fragmented contour elements as belonging to a single shape was still weaker than controls (data which support the masking findings reviewed above), leading to greater demands on higher-level visual brain regions involved in shape processing and distribution of attention. This view is also supported by data indicating that while improvements in perceptual organization can occur in schizophrenia with repeated exposure, these effects are often weaker than those observed in controls, and take longer to develop [reviewed in Silverstein and Keane (2009)].

The wider significance of perceptual organization dysfunction for understanding schizophrenia is that it may be manifestation of a more widespread disturbance in a fundamental cortical algorithm whose function is to detect consistent contextual/predictive relationships across space and/or time among incoming signals, and then to represent these relationships in new patterns of neural activity. In this view, the binding of features whose spatial relationships form the context for their inclusion in the same object representation is seen as analogous to the binding of words or concepts into coherent thought and linguistic structures, where the binding is based on context-appropriate meaning (Chechile, Anderson, Krafczek, & Coley, 1996; Fuster, 2005; Glezer, 1989; Glezer & Tsoukkerman, 1961; Logan & Zbrodoff, 1999; Phillips & Singer, 1997). This theme will be elaborated upon further in the section below on contextual modulation.

There has been disagreement over whether perceptual organization of all visual features is reduced in schizophrenia, or whether it is restricted to grouping of certain types of features. For example, a recent study suggested that it is only integration of orientation information (i.e., of lines with similar orientations), but not other cues (e.g., motion, size), that is reduced (Tibber et al., 2015). However, the findings of relatively normal perceptual organization in this study can be accounted for by characteristics of the subject sample. Patients in this study were clinically stable outpatients, many of whom had very low levels of symptoms (including levels within the normal range), and over half were diagnosed with the paranoid subtype. In such a sample, relatively normal perceptual organization would be expected, given past data that impaired perceptual organization is associated with high levels of disorganized symptoms (Keane et al., 2014; Silverstein, Bakshi, et al., 1998; Silverstein & Keane, 2011a; Uhlhaas, Phillips, Mitchell, & Silverstein, 2006; Uhlhaas & Silverstein, 2005a, 2005b), an acute psychotic episode [with performance normalizing as symptoms remit and functioning improves (Silverstein et al., 1996)], and non-paranoid status (Cox & Leventhal, 1978). Finally, there is much evidence on impaired integration of motion in schizophrenia using moving dots (where there is no orientation information) (Chen, 2011; Chen, Nakayama, Levy, Matthysse, & Holzman, 2003) and also with static stimuli (e.g., dots or asterisks) where orientation cues are not present (Cox & Leventhal, 1978; Rabinowicz et al., 1996; Silverstein et al., 2005; Silverstein, Bakshi, et al., 1998) or relevant (Silverstein et al., 1996). However, the possibility that integration of orientation cues is poorer than integration of other cues, and so present even in more clinically stable patients, needs further study.

Motion Perception. Abnormalities in the perception of motion in schizophrenia were noted as far back as 1908 (Diefendorf & Dodge, 1908). Over the past 100 years, increasingly sophisticated studies have replicated and extended these effects (Chen, Nakayama, et al., 2003; Levy, Holzman, Matthysse, & Mendell, 1993; Wang, Dobkins, McDowell, & Clementz, 2012). Motion perception problems in schizophrenia have been identified using various paradigms. One involves speed discrimination, where multiple studies have identified impairments. These studies indicate that the speed discrimination impairment is not related to the type of stimulus, and that it is most pronounced at intermediate speeds, where speed information is more important than position cues (which are most relevant to processing slow stimuli) or temporal frequency cues [reviewed in Chen (2011)]. However, questions remain about the extent to which schizophrenia patients’ difficulties in eye tracking tasks (including those not involving speed discrimination) reflect altered sensory input (e.g., abnormal feature processing or reduced motion sensitivity), poor top-down control over motor (e.g., eye movement) activity, faulty interactive connectivity between visual and motor areas, or other problems. For example, a significant relationship between speed discrimination and visual evoked potentials (which reflect the strength of the signal reaching V1 from the retina) suggested a magnocellular pathway contribution to impaired speed discrimination in schizophrenia (Kim, Wylie, Pasternak, Butler, & Javitt, 2006). Another study found relationships between backward masking task performance and motion perception in schizophrenia patients, again suggesting an early visual contribution (Brittain, Surguladze, McKendrick, & Ffytche, 2010). Data from electrophysiological work suggest, however, that abnormal performance reflects a reduced ability to use information about speed in higher-level cognitive processes (e.g., decision-making), rather than a primary perceptual difficulty (Wang et al., 2012). Another electrophysiological study by this same group demonstrated that only later waveform activity, reflecting parietal lobe activation (and therefore, presumably, attention), was related to poor motion direction discrimination performance in a group of people with schizophrenia (Wang, Brown, Dobkins, McDowell, & Clementz, 2010), a finding which replicated an earlier behavioral study also indicating a non-sensory basis to motion processing difficulties (Chen, Levy, Sheremata, & Holzman, 2004). On the other hand, while data indicate variety of extra-occipital (e.g., cerebellar, thalamic, temporal, parietal, frontal) contributions to eye tracking dysfunction in schizophrenia (Nagel et al., 2007), there is also reduced activity in occipital area MT (V5) during motion processing in people with schizophrenia (Lencer, Nagel, Sprenger, Heide, & Binkofski, 2005). Therefore, as recently noted by Chen (2011), the relative contributions of perceptual and non-perceptual factors to motion processing impairments in schizophrenia still need to be clarified.

Other data indicate that while perception of local motion, or the direction of a single stimulus, is not impaired, detection of coherent motion [i.e., the similar and simultaneous direction of movement of multiple elements, especially when embedded in randomly moving element noise (see Fig. 5)] is affected in patients with schizophrenia and their unaffected relatives (Chen, Nakayama, et al., 2003; Slaghuis, Holthouse, Hawkes, & Bruno, 2007). This suggests impairment in an integrative mechanism, similar to that observed in studies of reduced contour integration in noise (see section above).

Fig. 5
figure 5

Example of a stimulus from a coherent motion task. Reprinted from Schizophrenia Bulletin , Volume 35(1), Green et al., Perception measurement in clinical trials of schizophrenia: Promising paradigms from CNTRICS, 2009, p. 175, with permission from Oxford University Press

Three studies have investigated whether abnormal motion processing in schizophrenia is related to altered inhibition or surround suppression, a form of contextual modulation (see below). One of these studies found reduced surround suppression of motion (Tadin et al., 2006), one found normal performance (Yang et al., 2013), and one found increased suppression relative to controls (Chen, Norton, & Ongur, 2008). These differences may be due, in part, to different stimuli used in the studies (e.g., some emphasizing local motion and some global motion) (Chen, 2011). However, it is also likely that differences in patient samples affected the results. For example, the study that found normal surround suppression included clinical stable and relatively asymptomatic patients; the study that found reduced suppression found this mainly among patients with severe negative symptoms (i.e., with presumed prefrontal hypodopaminergia) (Davis, Kahn, Ko, & Davidson, 1991; Howes & Kapur, 2009); and increased surround suppression was observed in the sample where 37.5 % of the patients were inpatients at the time of testing, suggesting acute psychosis and striatal hyperdopaminergia.

In short, some data on motion perception impairments in schizophrenia suggest the possibility of early visual contributions, perhaps mediated by clinical state, as with contrast sensitivity and spatial frequency processing. However, there is stronger evidence for involvement of later processing contributions, including in visual attention and working memory, and in sensorimotor integration and control. There is also consistent evidence that integration of visual signals across space is impaired. Therefore, motion processing tasks occupy an important place in assessment of vision and cognition in schizophrenia. Because they involve integration across space (as in global/coherent motion) and time, and typically involve eye movements, they are particularly sensitive to sensorimotor integration and its interaction with attentional, memory, and cognitive control processes.

Effects of Prior Experience on Perception. People with schizophrenia have shown differences compared to controls in their susceptibility to visual illusions. In most cases, patients were less susceptible (and so, they perceived the stimuli more veridically than controls), but in a few cases they demonstrated increased illusion susceptibility (see below). Striking examples of reduced susceptibility come from studies of depth inversion illusions. In these cases, a concave surface is perceived as convex. A widely studied example of this involves the hollow mask illusion (see Fig. 6) (Gregory, 1970; Papathomas & Bono, 2004). This effect has been demonstrated multiple times in schizophrenia (including in unmedicated patients and prodromal patients) (Dima et al., 2009; Dima, Dietrich, Dillo, & Emrich, 2010; Dima, Dillo, Bonnemann, Emrich, & Dietrich, 2011; Emrich, 1989; Emrich, Leweke, & Schneider, 1997; Keane, Silverstein, Wang, & Papathomas, 2013; Koethe et al., 2009; Schneider et al., 2002). The generally accepted interpretation of the normal illusion effect is that stored knowledge and expectations about what a face looks like (e.g., it is convex) override the influence of the actual sensory information in the construction of the perceptual representation.Footnote 8 The task is thus seen as an example of Bayesian processing, in the sense that a strong convexity prior biases the interpretation of the evidence. In this view, a reduced illusion effect in schizophrenia, leading to a tendency to perceive the hollow mask more veridically than controls, is due to reduced connectivity between brain regions that normally provide top-down, experience-based feedback (e.g., frontal, parietal, and temporal areas) and sensory regions. This interpretation is supported by dynamic causal modeling of ERP and fMRI data from hollow mask studies of patients and controls (Dima et al., 2009, 2010, 2011). Importantly, these effects do not require binocular viewing (as the term “binocular depth inversion illusions ” used in some prior studies implied), the effect is not limited to faces and can be found with other objects and scenes, the effect obtains regardless of whether a real 3-D stimulus or pseudoscopic viewing is used, and the effects are most pronounced among patients with active positive symptoms (Keane et al., 2013).

Fig. 6
figure 6

Examples of stimuli used in the depth inversion illusion study by Keane et al. (2013). Subjects observed concave faces and scenes (i.e., all of the stimuli in the left panel) that were shown with or without color. Because of the concavity, the green fixation points were further from the observer than the surrounding regions (cheeks or landscape). A beige face was convex and served as a catch trial, to ensure subjects were not responding randomly. Reprinted from Journal of Abnormal Psychology, Volume 122(2), Keane et al., Reduced depth inversion illusions in schizophrenia are state-specific and occur for multiple object types and viewing conditions, 2013, p. 507, with permission from the American Psychological Association

Another illusion where schizophrenia patients have demonstrated reduced susceptibility is the Ebbinghaus illusion (see Fig. 7). In this illusion, the perceived size of a target circle is magnified when surrounded by smaller circles, and reduced when surrounded by larger circles. In several studies, we have shown that schizophrenia patients can discriminate which of two circles is larger to a similar degree as controls in the absence of surrounding context, but when context is present and misleading (e.g., the target circle that is larger is made to appear smaller by surrounding it with large circles), patients are more accurate than controls in their size comparisons. In conditions where context should be helpful to performance (e.g., the larger circle is made to appear even larger by surrounding it by small circles), patients do not show as much benefit as controls (Horton & Silverstein, 2011; Joseph, Bae, & Silverstein, 2013; Silverstein, Keane, et al., 2013; Uhlhaas et al., 2005; Uhlhaas, Phillips, Mitchell, et al., 2006). This effect has also been observed among young people at ultra high-risk for schizophrenia (Mittal et al., 2015). We have interpreted this effect as being due to reduced size constancy, and in particular, to reduced effects of prior knowledge about size and distance when judged in 2-D images. This interpretation is based on the following considerations: (1) in healthy children, susceptibility to the Ebbinghaus illusion develops over time, and is rarely present prior to 6 years old (Doherty, Campbell, Tsuji, & Phillips, 2010; Kovacs, 2000), suggesting that it is based on experience with the visual world; (2) it can be shown that the context circles serve as cues to the distance (in imaginary 3-D space) between the observer and the stimulus, and therefore that displays with smaller surrounds are interpreted as being at a far distance whereas displays with larger surrounds are interpreted as being closer (Doherty et al., 2010); and (3) in healthy adults, the size of near objects is consistently underestimated, and the size of far objects is often overestimated, depending on age (with the effects greatest in young adults) (Kavsek & Granrud, 2012). Of note, among schizophrenia patients, reduced sensitivity to the illusion is related to an increase in disorganized symptoms (Silverstein, Keane, et al., 2013; Uhlhaas, Phillips, Mitchell, et al., 2006), more normal performance is related to a higher level of depression symptoms, and the degree of illusion susceptibility is state-sensitive, with most patients showing significant normalization of performance from the beginning to end of inpatient treatment for an acute psychotic episode (Silverstein, Keane, et al., 2013).

Fig. 7
figure 7

Sample stimuli from the three conditions of an Ebbinghaus illusion task. The subject’s task is to indicate, on each trial (where stimuli from only one of the three conditions are presented), which target circle is larger. In this illusion, the perceived size of a target circle is magnified when surrounded by smaller circles, and reduced when surrounded by larger circles. In the examples in this figure, the target/inner circle on the right is slightly larger than the one on the left. Therefore, surrounding it with smaller circles amplifies the real difference (i.e., helpful context), whereas surrounding it with larger circles suppresses size perception (misleading context). From Silverstein et al., (2013) Effects of short-term inpatient treatment on sensitivity to a size contrast illusion in first-episode psychosis and multiple-episode schizophrenia. Front. Psychol. 4:466. doi: 10.3389/fpsyg.2013.00466. This is an open access article available at http://journal.frontiersin.org/article/10.3389/fpsyg.2013.00466/full

The studies reviewed in this section suggest a reduced vulnerability to illusions in schizophrenia, based on reduced effects of prior experience on interpretation of 2-D and 3-D stimuli. However, increased susceptibility to the Müller-Lyer illusion was observed among prodromal (but not chronic or first episode) patients (Parnas et al., 2001). And, Chen, McBain, Norton, and Ongur (2011) observed increased spatial frame illusion effects in schizophrenia patients. To the extent that schizophrenia patients are less susceptible to certain illusions but more susceptible to others, this implies multiple mechanisms at work [indeed, even though Müller-Lyer and Ebbinghaus effects can possibly both be explained in terms of amount of space surrounding the target (Nemati, 2009), this would not account for the pattern of results observed in studies of schizophrenia]. For example, while the Ebbinghaus illusion appears to involve perceptual organization and size constancy, the Müller-Lyer illusion may involve a form of sensory fusion in which the location of line endpoints and/or sharp angles are averaged to judge location.

The presence of multiple mechanisms is also supported by findings that extent of susceptibility to illusions purportedly involving surround suppression (see Fig. 8) (e.g., in motion, contrast, size, etc.) does not correlate significantly across illusions in healthy samples or patients (Tibber et al., 2013; Yang et al., 2013). And, patients with schizophrenia show impairments on some, but not all of these, with the evidence suggesting that those involving suppression at later stages (e.g., size, as in the Ebbinghaus illusion, and contrast) are more likely to be impaired than those involving earlier stages (e.g., luminance and contrast) (Tibber et al., 2013; Yang et al., 2013). A consideration to keep in mind, however, when interpreting results from surround suppression tasks is that performance may be more impaired in acutely psychotic patients (as in Silverstein, Keane, et al., 2013) and relatively normal in clinically stable and mildly symptomatic patients (as in Yang et al., 2013), and this will affect the magnitude of between-group differences, and possibly of inter-task correlations as well (via range restriction).

Fig. 8
figure 8

Example of surround suppression, with contrast, and its reduction in schizophrenia. (a) The small region at the center of the large circular patch is physically identical to the small patch at the top left but generally seems to be of much lower contrast (e.g., similar to the small patch at the bottom left) as a consequence of contrast gain control. (b) One can quantify this effect by plotting the probability that subjects said the central patch was higher contrast than a matching variable contrast reference patch. A typical control subject (green line) indicated that the central patch had a substantially lower contrast than it actually did (indicated by the shift in the green curve to lower reference contrasts). Data from a representative patient with schizophrenia (red line) indicated that they were not susceptible to the illusion and matched the contrast largely correctly. Reprinted from Current Biology , Volume 15(20), Dakin et al., Weak suppression of visual context in chronic schizophrenia, 2005, p. R823, with permission from Elsevier. Caption from (Butler et al., 2008)

Clarification of which illusions are most sensitive to schizophrenia, and when in the course of illness they are most sensitive, is an area where future research is needed, especially since many illusion effects can avoid generalized deficit confounds (because patients can outperform controls), and at least some are state-sensitive, and so they have potential as biomarkers. An issue to explore in this effort is the different roles of two types of inhibitory (GABAergic) interneurons on surround suppression and other forms of non-veridical perception. One, somatostatin (SOM) containing interneurons, contributes to surround suppression (Adesnik, Bruns, Taniguchi, Huang, & Scanziani, 2012; Phillips, Clark, & Silverstein, 2015; Wilson, Runyan, Wang, & Sur, 2012). The other, parvalbumin (PV) (but not SOM) containing interneurons have been shown to reduce the spiking rate of pyramidal cells (Atallah, Bruns, Carandini, & Scanziani, 2012; Wilson et al., 2012), and activation of PV (but not SOM) interneurons has sharpened orientation tuning and motion direction selectivity in pyramidal cells to which they were connected (Lee et al., 2012). Importantly, there are reduced numbers of both SOM and PV interneurons in schizophrenia (Wang et al., 2011). However, effects of alterations in each interneuron type on perception in schizophrenia and how this varies by clinical state have yet to be explored.

Contextual Modulation: An Integrated Model of Visual Impairment in Schizophrenia

The research reviewed in the previous sections indicates multiple forms of visual perceptual impairment in schizophrenia. To what extent can these be seen as manifestations of one, or a small number, of more basic dysfunctions? Here, I propose that the findings discussed above can be largely accounted for by a combination of illness (and medication)-related variability in sensitivity/gain, as discussed above (especially in the cases of contrast sensitivity and spatial frequency processing), and illness-related variability in contextual modulation (CM) . CM can be defined as influences that affect the sensitivity of a cell to its normal receptive field (RF) input, but that do not normally drive (i.e., lead to an action potential in) the cell itself (Phillips et al., 2015; Phillips & Silverstein, 2013). From a psychophysical perspective, this means that CM can change the threshold, slope (gain), or asymptote of the function relating neuronal input to output. In this way, CM can alter the precision of the estimate of the distal variable, by changing the width of the tuning function, without changing the peak of the function (Phillips et al., 2015). The concept builds on much work highlighting the distinction between driving and modulatory input (Gilbert & Sigman, 2007; Haider & McCormick, 2009; Lee & Sherman, 2010; Phillips & Singer, 1997; Salinas & Sejnowski, 2001), and is supported by studies demonstrating that pyramidal cells with different RF properties can be directly connected, and affect each others’ firing rates, while nevertheless retaining their sensitivity to the visual features to which they are tuned (Schummers, Marino, & Sur, 2002).

Three distinct forms of CM have been identified: modulation that (1) amplifies, (2) suppresses, or (3) synchronizes responses to RF input. The goal of each of these is to amplify processing of information that is relevant to the context within which it appears, and to suppress processing of information that is not relevant.

Modulation That Amplifies. Amplifying modulation has been demonstrated extensively in the flanker paradigm, where neural responses to short line or Gabor elements are increased in the presence of collinear flankers (including when these are placed outside of the RF of the cells signaling the target element) (Kapadia, Ito, Gilbert, & Westheimer, 1995; Mizobe, Polat, Pettet, & Kasamatsu, 2001). This is relevant to schizophrenia, as patients have demonstrated reduced facilitation effects on this task [e.g., Keri, Kelemen, Benedek, & Janka, 2005; Keri, Kiss, et al., 2005], although, for reasons that are not fully understood, this may be specific to higher spatial frequency display elements (Keane et al., 2014). Other aspects of perception, and its impairment in schizophrenia, can also be accounted for within the framework of amplifying modulation. For example, this concept can account for disambiguation of visual motion signals (by strengthening firing of motion signals that were identical to other concurrent motion signals) (Bayerl & Neumann, 2004), and thus may explain why coherent motion, but not local motion processing is disrupted in schizophrenia (Chen, Nakayama, et al., 2003). It is also relevant for perceptual organization, where it has been shown that neural activity is increased for visual features that are perceived as belonging to an object boundary (Flevaris, Martinez, & Hillyard, 2013), an effect that can occur in the absence of attention (Marcus & Van Essen, 2002). Contrast sensitivity, if viewed in part as a manifestation of enhanced processing at regions of luminance changes, can also be viewed as involving amplifying modulation (Butler et al., 2008). In addition, by increasing the precision of neuronal selectivity (e.g., by sharpening the tuning curve) of pyramidal cells, amplifying modulation is relevant to perceptual functions such as spatial frequency processing (see above), and backward masking deficits, which have recently been interpreted as due to overly broad occipital cortex neuronal tuning (Green et al., 2011). Indeed, regarding the latter, Herzog and colleagues have recently presented data suggesting that vulnerability to masking in schizophrenia is an aspect of a more general dysfunction in enhancing the processing of visual stimuli (Herzog & Brand, 2015; Herzog, Roinishvili, Chkonia, & Brand, 2013), a view supported by ERP abnormalities during target processing in a masking paradigm (Patterson et al., 1987). As noted above, in this view, even though patients and controls may not differ in no-mask performance in some paradigms (Green et al., 2003; Rassovsky et al., 2005), target processing is still thought to be suboptimal, leading to greater vulnerability to effects of the mask.

From a neurobiological perspective, amplifying modulation can be attributed to both inter- and intracellular mechanisms. The intracellular mechanism involves activity of NMDA receptors . NMDA receptors are voltage dependent: they alter a cell’s threshold for firing only when it is both partially depolarized and receiving lateral or feedback input. These modulatory inputs include inputs that have been statistically related to the target cell firing in the past (Phillips & Singer, 1997), and selective attentional enhancements dependent on current task relevance (Phillips et al., 2015). The role of NMDA receptors in CM in vision was shown in a study using a figure-ground segregation paradigm, where blocking these receptors in V1 led to marked impairment of figure-ground segregation, but without affecting the feedforward responses of pyramidal cells (Self, Kooijmans, Super, Lamme, & Roelfsema, 2012). There is also much evidence for NMDA receptor hypofunction in schizophrenia , and its effects on perception, cognition, and behavior (Moghaddam & Javitt, 2012; Phillips & Silverstein, 2003).

The intracellular mechanism involves different influences of two spike initiation zones on neocortical pyramidal cells. One is at the base (soma) of the cell, and is responsible for sodium spikes and action potentials associated with feedforward processing. The other is at the base of apical dendritic tufts (i.e., the branching structures at the ends of the long dendrites that emerge from the apex of the pyramidal cells), far from the cell soma (see Fig. 9). Tuft dendrites receive input from a variety of sources, including lateral and reentrant connections. Activity at the apical tuft has little effect on the rate of action potentials if there is no input to the soma. However, when there is both input to the soma and to the apical tuft, calcium spikes are triggered that travel to the soma, and this results in significant amplification of the cell’s firing rate, and can result in long-term changes to a cell’s sensitivity (e.g., to contextual and learning effects). In layer 5 cells, this mechanism has been called backpropagation-activated calcium spike (BAC) firing (Larkum, Nevian, Sandler, Polsky, & Schiller, 2009; Larkum, Zhu, & Sakmann, 1999; Major, Larkum, & Schiller, 2013; Nevian, Larkum, Polsky, & Schiller, 2007; Larkum & Phillips, in press; Phillips et al., 2015). While it has excitatory effects, it must be distinguished from excitation due to receptive field input to the neuronal soma. Importantly, BAC firing will be attenuated if NMDA receptors are blocked (Larkum, 2013). In layer 3 cells, tuft input amplifies sensitivity to driving basal inputs by a mechanism that is not dependent on BAC firing , but which remains dependent on NMDA receptors . The emerging data on the role of BAC firing and analogous mechanisms in CM suggest that abnormalities in these mechanisms may be involved in perceptual and cognitive changes in schizophrenia.

Fig. 9
figure 9

Layer 5 pyramidal cell. Realistic (a) and schematic (b) views. The apical (dendritic) tuft (in purple background in a; in red in b) receives amplifying or disamplifying inputs from neurons that are different from those that synapse onto basal dendrites or other dendrites close to the soma (in green background in a, in gray in b). When apical depolarization coincides with basal input, calcium spikes initiated by a site of integration near the top of the apical dendrite amplifies the cell’s response to its basal inputs (Larkum, Zhu, & Sakmann, 1999; Larkum, 2013), as shown in (b). The most studied mechanism by which AA is implemented in layer 5 cells is referred to as back-propagation activated calcium spike firing (BAC firing). In addition to these two main integration sites local integration takes place within both basal and tuft dendrites by the regenerative activation of NMDA receptors (NMDA-spikes). Disamplification does not affect the response of the cell to its receptive field input at the soma, but reduces amplifying effects. Figure 9(a) is reprinted from Nature Reviews Neuroscience, Volume 9, Spruston, Pyramidal neurons: dendritic structure and synaptic integration, 2008, p. 207, with permission from Nature Publishing Group. Figure 9(b) is reprinted from Behavioral and Brain Sciences, in press, Larkum and Phillips, Does arousal enhance apical amplification and disamplification? Reprinted with permission from Cambridge University Press

Modulation That Suppresses. Suppression of responses to RF input typically occurs when the stimulus driving the cell is surrounded by similar stimuli (Heeger, 1992; Kapadia et al., 1995). Suppression is involved in a number of aspects of vision, including sharpening of orientation tuning (Okamoto, Naito, Sadakane, Osaki, & Sato, 2009). It is therefore relevant to vernier acuity (judging the relative alignment of two edge elements), which has been shown to be reduced in schizophrenia (Keri et al., 2004). Suppression can also be observed in a flanker paradigm, either when the central target is flanked by similarly oriented elements at very close distances (Polat & Sagi, 1993) (at longer spatial distances, requiring connectivity over longer cortical distances, amplification, as described above, occurs), or by orthogonally oriented elements (Keri, Kiss, et al., 2005). In schizophrenia, inhibitory responding during a flanker task was shown to be intact (Keri, Kelemen, et al., 2005). However, there are multiple forms of impaired surround suppression in schizophrenia, where the effects of a completely surrounding context did not attenuate response to the central target to the same degree as among control subjects (Dakin, Carlin, & Hemsley, 2005; Tibber et al., 2013) (see Fig. 8). While in some cases, surround suppression is increased in schizophrenia (Chen et al., 2008), this has been explained in terms of reduced inhibition. There is also evidence that surround suppression changes in schizophrenia co-vary with clinical state, with abnormalities being most pronounced in more symptomatic patients (Dakin et al., 2005), and less pronounced to absent in clinically stable and relatively asymptomatic patients (Barch et al., 2012). In general, however, evidence for reduced surround suppression, and other forms of inhibition in schizophrenia [e.g., in illusion perception (Chen et al., 2011), in suppressing self-generated signals (Blakemore, Smith, Steel, Johnstone, & Frith, 2000)] suggests that suppressing modulation is often dysfunctional in this disorder. What remains to be explained is why patients are impaired in some, but not all forms, of this. As noted above, this may be due to different phases of illness in the patients tested in different studies, to different degrees of involvement of SOM and PV interneurons in different forms of inhibition, to effects of medication on interneuron function, and of course to heterogeneity within the patient samples we characterize under the umbrella term “schizophrenia.”

Activity at the dendritic apical tuft is relevant for suppression, as well as for amplification. There are inhibitory interneurons that specifically target apical dendrites, such as Martinotti cells , and elongated neuroglioform cells (Phillips, Submitted; Phillips et al., 2015). The effects of inhibition at apical dendrites would be to reduce the likelihood of amplification, since this input would restrict activity within the apical dendrite, which is compartmentalized from activity within the soma (Larkum, 2013). Therefore, while inhibitory activity at the apical dendrite reduces the likelihood of amplification, it would not prevent the cell from responding to its receptive field input. For this reason, suppression via apical dendrites must be distinguished from traditional forms of inhibition, and is better termed “disamplification” (Phillips, Submitted). Also, because amplifying and disamplifying influences are compartmentalized from input to the soma, these effects do not operate via integrate-and-fire point processors (i.e., those that sum their excitatory and inhibitory input and fire an action potential when a threshold is exceeded) as is assumed in classical models of gain control (Phillips, Submitted).

Synchronization. Much evidence indicates that perceptual organization is implemented and signaled largely by synchronization of neuronal oscillations (Phillips & Singer, 1997; Uhlhaas & Singer, 2006, 2010). These effects are strongest when there is weak feedforward activity [implying that synchronization involving reentry and recurrent feedback is especially important under these conditions (Engel, Fries, & Singer, 2001)]. It bears noting that the conditions under which patients with schizophrenia have particular difficulty with perceptual organization (i.e., when elements are spatially separated and top-down feedback is required) are exactly those where synchronization is paramount for its normal operation. Reduced synchrony has also been found to be related to impaired perceptual organization in schizophrenia (Uhlhaas, Linden, et al., 2006).

Many studies indicate that synchronization of oscillations within the gamma band are the most critical for the formation of neural assemblies (Fries, Neuenschwander, Engel, Goebel, & Singer, 2001; Spencer et al., 2003, 2004), especially for their feedforward signaling (Bastos et al., 2015), and for segregation of signal from noise (Buzsaki, 2006). Activity within this frequency band has also been found to be reduced in schizophrenia, with the extent of reduction related to extent of impaired perceptual organization and other disorganized symptoms (Grutzner et al., 2013). Importantly, it has been shown that PV interneuron activity is both necessary and sufficient for the generation of gamma oscillations (Cardin et al., 2009; Sohal, Zhang, Yizhar, & Deisseroth, 2009). Beta-band oscillations may be especially important for feedback (Bastos et al., 2015). These are also reduced in schizophrenia (Sun et al., 2014), and reduced synchronization of beta-band activity is related to impaired perceptual organization in schizophrenia (Uhlhaas, Linden, et al., 2006).

Currently, the role of activity at apical dendrites for synchronization is less clear than it is for amplification and suppression. However, given that the time course of amplification via apical dendrites is relatively long and that pyramidal cells in many different columns share common tuft inputs, it appears well suited to play a role in synchronization.

Summary: An Integrated View. We have seen that each of the visual impairments reviewed earlier (e.g., contrast sensitivity, spatial frequency processing, masking, perceptual organization, motion processing, and effects of stored knowledge on perception), in addition to related findings not reviewed but for which there exists much evidence (e.g., impaired vernier acuity ), can be understood as a manifestation of altered amplification, suppression, and/or synchronization—that is, of altered CM. This may also interact with more basic changes in gain, especially in the cases of lower level visual functions such as contrast sensitivity. This view allows for further integration of psychophysical and psychophysiological data with neurobiological evidence. For example, evidence reviewed in Phillips et al. (2015) suggests that PV inhibitory interneurons contribute to all three forms of CM. Amplification is related to their inhibition, suppression is related to their activation, and synchronization of the gamma rhythms they generate implements some forms of perceptual organization. There is also much evidence now for impairment in PV interneuron activity in schizophrenia , an overall reduction in their number, and the effects of these changes on neuronal oscillations and perception (and other cognitive functions) (Gonzalez-Burgos, Cho, & Lewis, 2015). This suggests that abnormalities in inhibitory neural functioning in general, and PV interneuron activity more specifically, could contribute to the range of perceptual impairments observed in schizophrenia. This is unlikely to be the whole story, however, since SOM interneurons are involved at least in surround suppression, and since we have seen that inhibition may be increased or decreased depending on the study sample or task. This issue will be addressed below. For now, however, what is clear is that a range of visual perceptual impairments can be accounted for in terms of impaired CM, and so tasks that measure visual impairments have the potential to be biomarkers of specific forms of CM.

There is also now general agreement that cortical sensitivity (i.e., gain), synchronization, and modulation are all tightly linked (Phillips et al., 2015). Evidence from computational modeling suggests that as activity levels increase, cells become more sensitive to temporal inputs occurring at the same time, and respond increasingly to synchronous rather than asynchronous input (Chawla, Lumer, & Friston, 1999). However, there are also optimal levels of activity and synchrony. Too little can lead to problems in sensory registration or overly weak gain, and too much can lead to the formation of aberrant cell assemblies (i.e., cases in which random and coincidental patterns of neural firing self-organize into stable networks) and therefore to increased noise and interference with processing of networks corresponding to the statistical structure of present reality or past experience (Olypher, Klement, & Fenton, 2006; Sun et al., 2013).

A factor that appears to be critical to overall level of cortical excitability is dopamine (Nitsche et al., 2010). One effect of dopamine is to modulate firing of PV interneurons (Sesack, Hawrylak, Melchitzky, & Lewis, 1998). Dopamine also has nonlinear effects on neural network function. Monte-Silva, Liebetanz, Grundey, Paulus, and Nitsche (2010) showed that whereas low and high dosages of L-Dopa (a dopamine precursor) reduced facilitatory and inhibitory plasticity, medium doses prolonged inhibitory plasticity (Monte-Silva et al., 2010). This nonlinearity may explain some of the effects of clinical state (i.e., movement from striatal hyperdopaminergia in acute states to hypodopaminergia in medicated and chronic patients) on perception in schizophrenia. However, it must be noted that dopamine has both excitatory and inhibitory effects, and these differ by receptor subtype. In addition, the only direct evidence for increased dopaminergic activity in the brain in schizophrenia is in the striatum, and striatal effects on the aspects of perception discussed in this chapter are generally not known (Ashby, Valentin, & von Meer, 2015; Vitay & Hamker, 2007). However, there is evidence that striatal dopamine modulates the strength of input from the lateral geniculate nucleus to the visual cortex,Footnote 9 which could explain the relationships between differences in striatal dopamine (excessive to reduced) in unmedicated first episode vs. medicated later episode patients on functions such as contrast sensitivity and spatial frequency processing (Van Opstal et al., 2014). This could have many downstream effects. For example, in schizophrenia there is reduced outflow of information from visual cortices (bilaterally) to the insula, which is thought to lead to reduced top-down feedback from the frontal lobe, and reduced salience processing (Palaniyappan, Simmonite, White, Liddle, & Liddle, 2013). Thus, for example, reduced dopamine availability, due to antipsychotic medication use and/or illness chronicity, could be involved in a range of factors such as weakened retinal signaling, reduced activity in visual cortex, reduced outflow from visual cortex, reduced top-down feedback, and a greater-than-normal reliance on (noisy) sensory representations (and the out-of-context associations they activate), as opposed to prior experience, during response generation. In contrast, excessive dopamine could lead to increased retinal signaling and hyper-processing of visual stimuli, increased afferent learning from LGN to V1 resulting in rapidly shifting and unstable receptive fields and orientation tuning, and the types of visual distortions noted earlier in this chapter, with their consequences for distractibility and delusion formation. Of course, much more research needs to be done to confirm whether these proposed sequences of neural and psychological events are occurring. Nevertheless, these are all testable hypotheses.

Glutamate is another neurotransmitter that has excitatory effects (Homayoun & Moghaddam, 2006), whose levels are altered in schizophrenia (Moghaddam & Javitt, 2012; Olney & Farber, 1995; Schobel et al., 2013), and that is involved in perceptual impairments in schizophrenia (Phillips & Silverstein, 2003). Glutamate is also the primary neurotransmitter in the retina (de Souza, Kalloniatis, Polkinghorne, McGhee, & Acosta, 2012). Much of the excessive glutamate in schizophrenia is thought to be due to NMDA receptor hypofunction, which has the effect of reducing activity at interneurons, with subsequent reduced inhibition at pyramidal cells leading to excessive excitation (Phillips & Silverstein, 2003). This scenario has been successfully modeled in humans and animals by NMDA receptor blockade by drugs such as PCP and ketamine (Moghaddam & Javitt, 2012). These drugs also produce perceptual organization deficits in nonpsychotic individuals (Neill, Joshua, Morgan, & Rossell, 2015; Uhlhaas, Millard, Muetzelfeldt, Curran, & Morgan, 2007) (which, in patients, are related to disorganized symptoms) [reviewed in Silverstein and Keane (2011a) and Uhlhaas and Silverstein (2005a)]. Dopamine and glutamate have interactive effects (Cepeda, Andre, Jocoy, & Levine, 2009), and at present it is not clear which neurotransmitter system (if either) is the source of the primary disturbance in schizophrenia, or whether this differs among subgroups of patients. However, other factors may also be involved in the heterogeneity in findings described previously. For example, as noted above, the extent of inhibitory regulation will vary depending on whether the basal soma or apical dendritic tuft is the source of inhibition. And, different aspects of inhibition will be affected depending on the extent to which PV or SOM interneurons are involved. Glutamate also interacts with arousal-related norepinephrine release to synergistically augment amplification and disamplification (Larkum & Phillips, in press), and increase synchronization of prioritized representations (Mather, Clewett, Sakaki, & Harley, in press). The roles of these interactions, arousal in general and norepinephrine in particular, in perceptual phenomena in schizophrenia, have yet to be clarified, however.

The currently available data do not allow for a precise mapping of visual perceptual impairments with changes in amplification, suppression, and synchronization as they occur in schizophrenia. However, enough evidence exists to generate testable hypotheses for each of the domains reviewed above, whose confirmation/disconfirmation would move the field closer to achieving this goal. These include: (1) contrast sensitivity in high-risk, acutely psychotic, and unmedicated patients reflects increased amplification (on top of increased gain) and reduced suppression related to increases in striatal and retinal dopamine and glutamate levels, whereas in chronically ill patients (possibly as a result of long-term medication use) reductions in contrast sensitivity reflect reduced amplification and gain, and reduced levels of these neurotransmitters; (2) changes in sensitivity to low and medium spatial frequencies will follow the same patterns observed for contrast sensitivity; (3) masking impairments reflect two processes, a vulnerability-related impairment in generating and synchronizing gamma-band oscillations, and illness-related impairments in amplification (causing weakened visual feature signals and object representations in LOC that are more vulnerable to disruption), and suppression or disamplification (which causes broader neuronal tuning); (4) perceptual organization impairments reflect reduced gamma and beta power and synchrony, and their severity should reflect an interaction between acuity of psychosis (related to NMDA receptor hypofunction and elevated striatal dopamine) and duration of illness (including poorer premorbid functioning), with illness chronicity effects being due to loss of occipital gray and white matter.Footnote 10 These anatomical changes should also be related to more broadly tuned feature processing and coarser representations, and so to findings such as reductions in vernier acuity (Herzog et al., 2013), orientation-specific surround suppression (Schallmo et al., 2013a), and pooling of orientation cues (Tibber et al., 2015); (5) changes from increased to normal surround suppression of coherent motion information should occur as patients move from acute to stabilized phases of illness, and this should be related to reductions in amplification and increases in suppression (and corresponding reduced positive symptoms and glutamatergic activity, and increased PV interneuron activity). In contrast, reductions in surround suppression should occur with increases in negative symptoms and illness chronicity. Because medication did not affect coherent motion detection in the absence of a surround suppression manipulation (Kelemen et al., 2013), coherent motion detection may reflect an impairment in perceptual organization based in reduced synchrony, in which case it should worsen over time with illness chronicity; and (6) effects of prior experience on perceptionFootnote 11 (e.g., as with the hollow mask illusion ) should increase as patients move from the acute to stable phases of illness [as shown in Keane et al. (2013)], and this perceptual change should be associated with increased inhibition of LOC output by frontal and parietal activity over the course of symptom remission. This hypothesis is based on the finding of frontal–parietal inhibition of LOC output in healthy controls during a hollow mask task, but a reduction in inhibition, and relatively stronger LOC activity (and more veridical perception of the hollow mask), in schizophrenia patients (Dima et al., 2009). Changes in this pattern have not yet been studied longitudinally with treatment, however.

Finally, it must be noted that CM is not limited to either vision in particular or to perception in general. At the neural level, it is a widespread operation that is found throughout the cortex, and it is involved in contextual disambiguation in all sensory domains (e.g., in object and word perception), selective attention (which can be viewed as the organization and segregation of inputs, and amplification of one and suppression of others), cognitive control (e.g., as with the Stroop task , where suppression of pre-potent responses is required), and flexible selection and coordination of actions (Phillips et al., 2015). Therefore, one of the advantages of studying vision in schizophrenia is that many tasks can be viewed as probes of this general, CM, process. But unlike traditional neuropsychological tests, and many paradigms from cognitive psychology, tests of visual function have been developed (to assess functions such as backward masking, surround suppression, and perceptual organization) that avoid generalized deficit confounds (Knight & Silverstein, 1998, 2001; Silverstein, 2008). Because specific processes can be isolated very well in many perceptual tasks, these tasks are especially good methods for studies of issues such as schizophrenia development prediction, medication mechanisms, mechanisms involved in relapse and recovery, and other dimensions of etiology, pathophysiology, and cognition. It must be noted, however, that while the perceptual and cognitive functions noted above can be conceptualized as involving different levels and combinations of the three forms of CM, whether or not performance across tasks measuring these processes (e.g., selective attention and perceptual organization) is significantly correlated has yet to be determined. Different variations on the theme of CM may be differently affected across patients and within a person’s course of illness.

What Does the World Look Like for People with Schizophrenia ?

Having now reviewed data on subjective visual distortions, as well as much laboratory psychophysical, imaging, and psychophysiological data, we can now circle back to the question of in what ways does the world look different for people with schizophrenia. There can of course be no single answer to this question, since schizophrenia is a heterogeneous category, and for any single person, subjective experience will differ across the acute and stable phases of illness. Nevertheless, this is an important question.Footnote 12 If we are to truly understand the patients we work with, it would seem that having an understanding of the way their visual experience may be altered, and what thoughts and feelings this engenders, would be quite important. To date, there has been very little work on this question. However, enough is known that a start can be made. For example, much clinical and research evidence indicates that prior to treatment for a first episode of schizophrenia, and often during acute psychotic relapses, patients are characterized by sensory gating impairments and increased sensitivity to contrast, and experience the world in overly intense or distorted ways (e.g., overly bright colors, metamorphopsia, see Table 1). An excellent summary of the often hyper-intense and distorted subjective experiences of patients at the onset of their initial psychotic episode can be found in Chapman (1966). In chronically ill patients, however, a different picture emerges, one that is often characterized by reduced stimulus intensity, and increasing difficulties with psychic fragmentation. This change with illness progression may be, as noted above, related to reductions in dopamine and glutamate activity with illness progression, a process that is often accelerated by dopamine-receptor blocking antipsychotic medications. Progressive loss of occipital lobe gray and white matter may also contribute. The contrast sensitivity reductions found in treated patients led Kantrowitz et al. (2009) to propose that chronic schizophrenia patients see the world more dimly than other people. Progressive worsening of perceptual organization with illness chronicity would be expected to have additional effects, such as a weakening of gestalt formation, leading to a need to scan objects, faces, and scenes, to a greater degree than before, to assess their significance. This could also lead to normally unimportant objects in scenes being more salient (i.e., capturing more attention) than usual.

An important consideration in trying to understand the visual experience of people with schizophrenia is that we must move beyond understanding what the world looks like, to appreciating what it feels like. This is because, as noted above, changes in visual experience are often associated with changes in mood, and feelings of strangeness, alienation, and impending crisis. It is my belief that clues to the visual experience of people with chronic schizophrenia can be found in the late art of the American landscape artist George Inness (1825–1894). In many of his paintings, especially those after 1880, several features are notable, including: reduced contrast and an overall dimming of luminance, weakening of contours and increased blurring, ambiguous object form, patches of color that are found outside of their expected object boundaries, scratches applied to canvases that can imply forms that are not in the painting, placement of objects that do not cohere with other objects in the scene, the use of paths that create ambiguity between foreground and background and that require the viewer to order and reorganize each scene, and, related to this, an often unclear organization in the image (DeLue, 2004) (see Fig. 10). Regarding the latter, rather than there being a clear focus that guides the eye through the image (as is typical in representational art), there is often an “irrelevant” object in the center which forces the viewer to scan the image in search of the most significant details. Inness’ paintings were often criticized by contemporary art critics as being familiar (in the sense of being large landscapes) yet strange and disturbing (in the sense of the features noted above) (DeLue, 2004). This combination of familiarity and strangeness creates in the viewer a sense of the uncanny, as well as one of discomfort, gloom, and strong emotion—feelings often described by patients. It is interesting that Inness, who did not have a psychotic disorder, deliberately developed this style in order to promote “spiritual sight” by forcing the viewer to not rely on overlearned habits of seeing (Bell, 2006; DeLue, 2004). In doing so, however, he also created representations of visual worlds that are “adjusted” to match several of the most prominent visual changes in schizophrenia, especially in cases of chronic illness (e.g., reductions in contour integration, contrast sensitivity, and contextual disambiguation).

Fig. 10
figure 10

George Inness. Summer Montclair (New Jersey Landscape) (1891). Note the lowered contrast, blurred contours, ambiguous forms, inclusion of two largely leafless trees oddly juxtaposed with a group of heavily leaved trees and centered in the image, and lack of clear organization and focus in the image. Reprinted from R. Z. DeLue, George Inness and the Science of Landscape, 2004, Plate 15, with permission from the University of Chicago Press

Conclusions and Future Directions

Although psychology has had a long history of studying perceptual and cognitive processes, psychometric measures of these functions have never become part of the routine diagnostic workup for schizophrenia, even though this has been proposed in the past (Weiss, 1989, 1990, 1992), and even though the utility of behavioral and psychophysiological measurement for understanding and treating schizophrenia was demonstrated long ago (Jung, 1907/1960). In general, an approach to schizophrenia that is grounded in psychological processes has not emerged in clinical psychiatry. This has led to a split between research practice and diagnostic practice. Even within the field of cognitive neuroscience, it can be argued that the development of tools within neuroscience has outpaced our understanding of the psychological and computational processes that need to be studied, leading to suboptimal progress toward an integrated understanding of the disorder within the context of mass accumulation of individual research findings. Therefore, two important tasks continue to be: (1) understanding the fundamental processes that are impaired in schizophrenia; and (2) finding ways to detect them that can be useful (i.e., informative and feasible to use) in routine clinical practice. While the impact of efforts toward these goals will be to some extent a function of the extent to which heterogeneity within the current category of schizophrenia can be better understood, progress toward these two goals can also accelerate this third effort by establishing the psychological and neurobiological bases for the way we view heterogeneity within schizophrenia.

Given this general state of affairs, a question becomes—to what extent can assessment of vision in people with schizophrenia accelerate progress? We have seen that there are many forms of visual impairments in schizophrenia, and many of them can be measured reliably and rapidly in typical office settings (ERG takes only a few seconds, and psychophysical tasks involving illusion perception can often be completed in under 10 min). I have argued, however, that many of the impairments observed in experimental studies can be understood as changes in the fundamental process of CM, and in particular, in combinations of its three subprocesses of amplification, suppression, and synchronization, possibly as overlaid on changes in level of gain. In this view, each of the many laboratory demonstrations of visual processing impairment in schizophrenia reflects a single, task-constrained manifestation of “the real problem.” Whereas we have learned much from the plethora of experimental paradigms that have been developed since the cognitive revolution of the 1960s, and the “decade of the brain” in the 1980s, we should at least consider that even more progress might be made by focusing our search for biomarkers around an emerging understanding of core processes. As implied above, developing cognitive theories and computational models can help bridge the gap between biology and mind. Fortunately, several of the currently popular models for understanding visual impairments in schizophrenia are based in rigorous computational models that are biologically realistic [e.g., Friston (2010); Kay and Phillips (2010); Phillips and Singer (1997)].

It has often been said that maximum progress will be made through interdisciplinary collaboration. Out of multidisciplinary efforts, new ideas and scientific paradigms emerge, and this has been, and continues to be the case, in schizophrenia research (Silverstein, Moghaddam, & Wykes, 2013). However, it can be argued that the majority of research in schizophrenia (and other fields) is not as generative of progress as it could be, due to several factors. One is of course the complexity of schizophrenia itself, a complexity that can seem to consign the study of any single phenomenon (e.g., vision, cognitive control, COMT) to a miniscule degree of importance. A second factor is the socio-cultural environment of academia, which requires high rates of individual faculty member productivity (in terms of published papers and acquisition of grant funding) as a condition of sustained institutional support. This scenario creates a situation in which faculty members can increasingly become focused on (and reinforced by their institutions for) perpetuating their own research programs and level of extramural funding, with increasingly less regard for what is actually needed in the long term (i.e., new ideas and collaborative, integrative, and large-scale efforts) to solve scientific problems (Kressel, 1990) [see also comments in Lykken (1991) about researchers building their own “castles in the sand” at the expense of cumulative science]. We must be vigilant to avoid the situation where we look back and find that each study, each publication, each talk at a research conference, and each grant, etc., was “simply one stroke in the plan, one thread in the fabric, and the plan was called the intellectual activity and the fabric was called the education industry and neither the whole nor any of the separate specialties had the slightest value whatever” (Hesse, 1971, p. 216). Without continued progress and radical re-visioning, we run the risk that our so-called scientific revolutions are simply just applications of new terminology to existing questions, and so more socio-rhetorical phenomena than true scientific advances (O’Donohue, Ferguson, & Naugle, 2003).

With these thoughts in mind, I propose the following ten questions as relevant vectors in a research agenda for vision in schizophrenia :

  1. 1.

    To what extent can measures of visual function serve as indices of the integrity of specific canonical cortical computations (e.g., divisive normalization, CM) and thereby help clarify aspects of brain function in other cognitive domains, phenomenology, and behavior? While work has been done examining perceptual organization impairment as an index of the computational failures involved in other forms of disorganization, there is much more work that needs to be done, especially regarding other visual processes.

  2. 2.

    Which visual abnormalities are trait-like and associated with (forms of) schizophrenia (including its genetic liability factors); which are related to severity of positive, negative and/or disorganized symptoms; and which are related to other symptoms (e.g., depression, anxiety), experiences (e.g., trauma history, smoking), or dimensions of functioning (e.g., arousal, HPA axis activity, mood) that cut across current diagnostic categories, as emphasized by the NIMH RDoC initiative (Cuthbert & Insel, 2010)?

  3. 3.

    To what extent can the assessment of visual function via laboratory tasks, and/or clinical assessment of visual distortions, inform the prediction of risk for outcomes such as conversion to psychosis, relapse, and treatment response?

  4. 4.

    Can screening for retinal and ocular abnormalities be useful in identifying illness risk and progression?Footnote 13

  5. 5.

    To what extent does the protective effect of congenital blindness on the development of schizophrenia (Landgraf & Osterheider, 2013; Leivada & Boeckx, 2014; Silverstein, Wang, & Keane, 2012; Silverstein, Wang, & Roche, 2013), which is stronger than the inverse relationship between rheumatoid arthritis and schizophrenia (Eaton, Hayward, & Ram, 1992; Gorwood et al., 2004; Mors, Mortensen, & Ewald, 1999; Oken & Schulzer, 1999), help inform our understanding of the importance of abnormal visual input in the development of the disorder, and about aspects of brain reorganization that could inform future prevention and treatment efforts?

  6. 6.

    Can visual disturbances in schizophrenia be treated, and if so, what methods would be useful, and what would the effects be in terms of higher-level cognition (e.g., visual working memory, reading) and symptoms?

  7. 7.

    To what extent do alterations in BAC firing, or in other mechanisms by which tuft input amplifies pyramidal cell output, account for each of the visual disturbances in schizophrenia discussed above; and, to what extent are these due to changes in physiology (e.g., activity at NMDA receptors) vs. anatomical changes (e.g., reduced dendritic branching) (Black et al., 2004; Glantz & Lewis, 2000).

  8. 8.

    What other mechanisms are involved in CM, and in the visual disturbances in schizophrenia. Recently, for example, intra-dendritic CM has been reported (Behabadi, Polsky, Jadi, Schiller, & Mel, 2012), although its relevance for amplification, suppression, and synchronization is not yet clear.

  9. 9.

    What are the differential effects of dopaminergic, glutamatergic, and noradrenergic activity on sensitivity of neurons to activity in their receptive field vs. BAC firing, and how do these differences affect visual function in schizophrenia?

  10. 10.

    What determines whether a brain region will be characterized by hyper- or hyposynchronization in schizophrenia (and what are the effects of this variability on visual function in the disorder?) [e.g., Rivolta et al. (2014)].

Generating answers to these questions can accelerate our understanding of schizophrenia (both as a disorder and in the cases of individual patients), similar to the way the intense focus on prefrontal cortex functioning has done over the past 30 years. Ultimately, however, the utility of a focus on vision in diagnosis, treatment, and research in schizophrenia is an empirical question. Therefore, it is my hope that this chapter and other writings on this topic will inspire young (and other) investigators to include self-report and laboratory measures of vision and visual processing in their research and treatment efforts. Given that treatment outcomes have barely changed in many years, despite much research and the introduction of many new interventions (Insel, 2010), there would seem to be little to lose and much to gain.