Position

The difficulty of explaining lightness or brightness perceptions is compounded by the absence of any cortical region dedicated to processing luminance. And although a relatively close link exists between light intensity and the rate of action potential generation by neurons in the retina and visual thalamus, this relationship breaks down when neurons are tested in the "higher" processing stations of the visual system. Neurons in the visual cortex generally respond only weakly to changes in light intensity. Yet these neurons represent the more complex aspects of stimuli, as evidenced by the selective receptive field properties of complex and hypercomplex neurons, and even neurons that respond to particular objects (see Chapter 7). Although neuronal activity at these higher cortical levels must be generating what we see, the representation of luminance—the most basic of the visual qualities—seems to have been lost in the shuffle.

Figure 8.6 A stimulus pattern that elicits perceptual effects that, as described in the text, undermine explanations of lightness/brightness contrast based on the properties of neurons at the input level of the visual system. The key indicates the physically identical target patches of interest. (From Purves and Lotto, 2003)

How, then, could an empirical perspective ever hope to make sense of the perceptually perplexing and physiologically messy relationship between luminance and lightness or brightness? Because explanations of perceptual qualities in wholly empirical terms are a recurrent theme in the chapters that follow, it may help to define how, in principle, this strategy could account for the strange way that we perceive brightness and lightness. The standard stimulus in Figure 8.7A (and Figure 8.1) is, on empirical grounds, consistent with the two identical targets being physically similar surfaces under the same amount of light ( Figure 8.7B), or the two physically different surfaces under different amounts of light ( Figure 8.7C). In a wholly empirical framework the frequency with which these possibilities occur in human experience would have influenced the evolution of the relevant visual circuitry accordingly, creating neural connectivity in the visual brains of successive generations that reflected the consequences of trial-and-error responses to natural stimuli. Some members of each generation, through inheritance and random genetic variation, would possess visual circuitry that generated perceptions that facilitated behavioral responses to the sources of stimuli a little more effectively than the circuitry in other members of the cohort. These individuals would, on average, be slightly more successful in life and reproduce a little more effectively. As a result, neural connections that linked the relevant visual stimuli to operationally useful perceptions and behavior would gradually wax in the visual brains of the population. Perceptions arising in this way would correspond not to any particular feature of the stimulus, but to the lightness or brightness perceptions that had proved the best link to successful behavior in the past. By contending in this way with the inverse problem, identical targets in different surrounds in Figure 8.7A would look differently light or bright because the perceptions of the sources needed to generate successful behavior would necessarily be different (compare Figures 8.7B and 8.7C).

To make the biological rationale for this way of generating perceptions more concrete, think of a collection of surfaces—such as pieces of paper—having a range of physical compositions, with some reflecting more light and some less light when measured under the same illumination. If vision's goal is to distinguish objects that are physically the same or different in any circumstance, it would be of little or no use to perceive the luminance values generated by the surfaces of the papers. For example, imagine cutting one of the papers in half and placing the two halves at random somewhere in the room you are sitting in; the identical halves would likely end up in places where they would be illuminated differently and would therefore return two different luminance values to the eye. Conversely, the luminance of two physically different papers could be the same under different conditions of illumination, again making their luminance useless as a means of distinguishing their actual characteristics. Given that it is biologically useful to see the two papers as members of the same class (imagine them to be edible, and thus something of biological value), human ancestors who made this distinction a little better because of their slightly different visual connectivity would reproduce a little more successfully as a result of this advantage. Eventually, the visual system of humans or other species would discriminate surfaces in different contexts quite well on this empirical basis, despite the inherently uncertain meaning of luminance values in retinal images.

Figure 8.7 Understanding the relationship between luminance and lightness/brightness in empirical terms. A) A standard simultaneous lightness/brightness contrast stimulus, such as the one in Figure 8.1. (B) and (C) show the two different physical situations that would both have produced the stimulus in (A). See text for further explanation. (From Purves and Lotto, 2003)

(A) Standard simultaneous brightness co ni rast st imu fus

(A) Standard simultaneous brightness co ni rast st imu fus

(C) Different surfaces under different rlEurninates

Assuming that this conception of vision made sense, the next step for us was to test its validity in a serious way. At least two possibilities came to mind. Because many stimuli elicit peculiar lightness or brightness effects, we could test whether each of these particular puzzles has a plausible explanation in empirical terms. Alternatively, we could test this interpretation of vision more directly by asking whether human experience with luminance values in retinal images in relation to objects and conditions in the world accurately predicts the lightness or brightness people end up seeing in response to any given stimulus. Pursuing these goals during the next few years depended critically on Beau Lotto, a postdoc who arrived in the lab in 1998. I had been lucky the day Jeff Lichtman appeared in my office in 1974, and I was equally lucky when Beau showed up. Like Tim Andrews, Lotto had done his doctorate in developmental neurobiology in the United Kingdom, and had come with the idea of pursuing some developmental issue in vision. But by the time he arrived, my interest in development had faded and the effort directed at visual perception was growing. After some initial discussion about what to do, he threw himself into the work on perception, and he had all the skills needed to push things along in new and imaginative ways. Similar to Lichtman, Lotto grasped the nub of conceptual or technical problems right away, and he had the intelligence and tenacity to solve them. He was (and is) as much an artist as a scientist, and his ability to make visual tests and demonstrations was as good as Williams's, who was about to leave to start a company (Williams eventually became a successful entrepreneur in graphics and iPod and iPhone applications, many of them related to medical education and the brain).

Because we had no idea how to acquire or analyze data that could serve as a proxy for human visual experience with luminance or anything else, we started with the easier task of showing that some of the most perplexing brightness effects had plausible empirical bases. The first phenomenon we studied was known as Mach bands (Figure 8.8). Ernst Mach was a nineteenth-century German physicist whose name has been immortalized in the unit given to the speed of sound (Mach 1), and whose philosophical opposition to the reality of atoms helped set the stage for Einstein's paper on Brownian motion in 1905. Much like his contemporaries Helmholtz and Maxwell, Mach was deeply interested in vision, and in 1865 he described a band of increased darkness at the onset of a luminance gradient and a band of increased lightness at its offset that had no physical basis ( gradient here means a gradual transition).

Figure 8.8 Mach bands. The stimulus that elicits this effect is shown in the upper panel (A); the middle (B) and lower (C) panels show, respectively, the physical (photometric) and perceptual profiles of the stimulus. A luminance gradient between a uniformly lighter and darker region causes people to see a band of maximum lightness at position 2 and a band of maximum darkness at position 3. The luminance values in the middle panel are those on the printed page in (A) and show that these bands have no basis in physical reality. (After Purves and Lotto, 2003)

Another example that involves gradients is the Cornsweet edge effect ( Figure 8.9). Tom Cornsweet, an accomplished psychologist, instrument designer, and psychophysicist, described the effect that bears his name about a century after Mach's report. In this case, a gradient from gray to black, when opposed to a gradient from gray to white (forming the "edge" in the name of the effect), causes adjacent gray territories identical in luminance to look different. The surface adjoining the gradient going from gray to black looks darker than the surface that abuts the gradient going from gray to white.

Given their physical and mathematical bent, both Mach and Cornsweet proposed that these odd effects are incidental consequences of physiological interactions in the retina, and elaborated detailed models (and complicated equations) to indicate how retinal processing could lead to the perceptions. Despite a lot of further work by psychologists and psychophysicists, these theories remained in the literature as the best explanations. Lotto and I argued that both Mach bands and the Cornsweet edge effect were more likely to be based on the link between the luminance values in the stimuli that Mach and Cornsweet devised and the real-world sources of these stimuli that observers would always have experienced. In the case of Mach bands, we showed that the curved surfaces that humans routinely see produce light gradients with actual regions of increased and decreased luminance at their offset and onset; these are the familiar highlights and lowlights produced by the physical effects of curvature on reflected light ( Figure 8.10A). In the same vein, we showed that when the elements of a Cornsweet edge stimulus are present in natural scenes, the adjacent territories do return physically different luminance values to the eye ( Figure 8.10B).

The empirical interpretation of these perceptual phenomena is that human experience interacting with the natural sources of the stimuli that Mach and Cornsweet described would have influenced the evolution of visual circuitry according to the success or failure of the behavioral responses to these stimuli. As a result, the lightness and brightness values generated by visual system circuitry would, over evolutionary time, gradually become determined by the relative success of that perception in dealing with the relevant sources. In consequence the lightness/brightness values seen would reflect this operational success and not luminance values per se, explaining the effects that Mach and Cornsweet had described.

Figure 8.9 The Cornsweet edge effect. The stimulus is shown in the upper panel (A); the middle (B) and lower (C) panels show the physical and perceptual profiles of the stimulus, as in Figure 8.8. Despite the identical luminance values of the territories adjoining the two gradients (2

and 3), the entire territory between 1 and 2 looks darker than the territory between 4 and 3. (After Purves, et al., 1999)

Although Lotto and I were pleased with what we took to be clever empirical explanations of phenomena that had puzzled people for a long time, the general response of vision scientists was that these accounts could not be taken seriously because they were not linked to the enormous body of physiological information about visual neurons that everyone assumed would soon explain perception. This complaint was frustrating: If we were right, the relationship between the properties of visual neurons and perception could never be explained in the logical framework that everyone had assumed would give the answers to such puzzles. What we see is certainly caused by light stimuli that activate neurons in the primary and higher-order regions of visual cortex. But if a wholly empirical strategy is the brain's operating principle, then the visual system was a welter of neuronal connections built up historically over evolutionary (and individual) time according to all the factors that go into the determination of successful behavior. Not surprisingly, people whose careers were founded on a logically understandable hierarchy of sensory processing did not welcome this different idea of how brains work.

Figure 8.10 Real-world sources of the Mach and Cornsweet stimuli. A) Photograph of an aluminum cube in sunlight. Physical highlights and lowlights adorn the curved surfaces of real-world objects, as indicated in the accompanying photometric measurement. Note the similarity of the luminance profile here to the perceptual profile of Mach bands in Figure 8.8. B) As shown in this diagram of two cubes with curved surfaces made of different material and under different amounts of illumination, the components of the Cornsweet stimulus also arise routinely in natural scenes. When they do, the flat surfaces made of different material that abut the curvatures that generate the luminance gradients have different luminance values. (From Purves and Lotto, 2003)

In an empirical conception of vision, making sense of the neurophysiology underlying perception would ultimately depend on understanding the evolutionary history of the human nervous system in terms of behavior. And that goal seemed impossible, at least in the short term. If we had a proxy for visual experience with some limited aspect of the natural world—such as human experience with luminance—we could test whether that body of empirical information predicted some of the lightness and brightness perceptions that we had been studying. Although we began to think more about this possibility, it was not clear how to proceed in 1999. The easier path was to add more evidence to our claims by examining the perceptions elicited by some other visual quality in empirical terms, asking if other longstanding puzzles could be reasonably accounted for in this way. Color seemed to be the visual quality best suited to this purpose.

9. Perceiving color

Color is a fascinating perceptual quality, and almost everyone interested in vision during the last few centuries—Newton in the seventeenth and eighteenth centuries; Young, Maxell, Helmholtz, Mach, and even Goethe in the nineteenth century; and a host of investigators in the twentieth century—has wrestled with it. The extraordinary effort to rationalize color vision is ironic because color vision is not very important biologically: Many animals (dogs and cats, for example) have little color vision, and people with most forms of "color blindness" have only minor problems getting along in life.

What is color vision, and why have some species (including ours) gone to the trouble of evolving it? The perceptions of lightness and brightness discussed in the last chapter concern what we see in response to the overall amount of light that reaches the eye; by definition, these qualities are the perceptual responses elicited by this physical aspect of light stimuli. Seeing in shades of gray that range from black to white occurs when the energy in stimuli is more or less evenly distributed across the light spectrum (Figure 9.1A); seeing in color is the perceptual quality generated in us, and presumably many other visual animals, when the energy in a light stimulus is unevenly distributed, as Isaac Newton first showed in the late seventeenth century ( Figure 9.1B). However, Newton recognized that neither light nor objects are colored. Our brains generate the colors we see for reasons of biological advantage, just as brains make up the qualities of all our other perceptions. If you have doubts about this assertion, consider the perception of pain. The sensation we perceive when we accidentally touch a hot stove is not a feature of the world but a sensory quality that leads to useful behavior. The entirely subjective sense of pain makes us remove the hand before much damage is done and teaches us to be cautious of such objects in the future.

Figure 9.1 White light spectra and their decomposition. A) The spectra of daylight, fluorescent light, and light from an ordinary light bulb (a tungsten filament) do not elicit the perception of color when reflected from surfaces that preserve their relative uniformity. Although the energy in these several sources varies across the spectrum, it is distributed evenly enough so that all three human cone types are stimulated to about the same extent. B) As Newton showed, "white light" can be broken up by a prism into component spectra. The components elicit perceptions of color because they stimulate the three cone types in the human retina to different degrees. (From Purves and Lotto, 2003)

The advantage of color vision is more subtle and less important than distinguishing objects on the basis of luminance, which explains the relatively innocuous consequences of color-deficient vision. Nevertheless, the advantage of seeing color is not hard to appreciate. Although any visual animal can distinguish boundaries that depend on relative amounts of light, animals with little or no color vision are unable to distinguish surface boundaries that do not differ in luminance but only in the distribution of spectral power. A visual animal that can identify boundaries based on both these qualities of light by seeing colors in addition to light and dark will be better able to distinguish and classify objects in the environment, such as predators or prey. Animals that have not evolved color vision presumably did not need this extra visual aid as much as the animals that did. As might be expected, animals that don't see color well are typically nocturnal, or exist in an ecological niche that does not offer much need for this real but relatively modest visual advantage.

By the end of the nineteenth century, Young, Helmholtz, and Maxwell had argued convincingly that the spectral differences that Newton first described generate color perceptions by differentially activating three different receptor types. In the twentieth century, the postulated receptors were found to be three cone types distinguished by a different light-absorbing pigment in each. Helmholtz's idea that these three receptor types determine the colors we see is called the trichromatic theory of color vision. Sensible as his idea seems, the trichromatic theory is only half right: The three human cone types certainly play a critical role in color vision (witness the deficient color vision that occurs when one or more of the cone pigments is altered or missing), but they are only part of story.

The idea that stimulation of the three cone types is a sufficient explanation of color vision ran into trouble almost right away. Ewald Hering, Helmholtz's contemporary and sometime nemesis, pointed out that some aspects of the colors we see are not readily understood in terms of retinal receptor types. Hering's main objection was that humans with normal color vision perceive red to be an opponent color to green, and blue to be an opponent color to yellow. Although observers can see or imagine a transition from red to yellow through a series of intermediates without entertaining any other primary color perception, no parallel perception—or conception—exists for getting from red to green without going through blue or yellow; and there is no way of getting from blue to yellow without going through red or green. Moreover, humans perceive a particular red, green, blue, and yellow to be unique, meaning that we see one particular value in each of these four color categories as having no admixture of any other colors (see Figure 10.1). This unique quality is different than the way we see other colors. There is, for example, no unique orange or purple, orange always being seen as a mixture of red and yellow, and purple as a mixture of blue and red. Because the different activation of the three retinal cone types by light offers no explanation of these perceptual phenomena, trichromacy theory is, Hering argued, an incomplete account of how color sensations are generated. Even today there is no consensus about why we see these four "primary" color categories.

The central question that has intrigued everyone from neurobiologists to philosophers is why we see the colors in the way we do. Coming up with a plausible answer has been difficult, primarily because color vision entails so many puzzles. In addition to Hering's concern about the perception of opponent colors, it has long been known that color perceptions don't match the physical measurements made by a spectrophotometer. Similar to the lightness and brightness effects described in Chapter 8, when two target patches return the same spectrum to the eye but are surrounded by regions that reflect different distributions of light energy, the color perceptions elicited by the two targets are changed ( Figure 9.2). This phenomenon is called simultaneous color contrast. Just as puzzling are effects that seem the opposite of color contrast: the context surrounding two patches that reflect different spectra to the eye can elicit similar color perceptions, a phenomenon called color constancy ( Figure 9.3). Moreover, a banana looks yellow and an apple looks red whether the fruits are observed in the "bluish" light from the sky, the "reddish" light of sunset, or the "yellowish" light that comes directly from the sun. In each circumstance, the distribution of light energy reaching the eye from the surfaces of the same objects is quite different. These puzzling effects underscored Hering's argument that color vision cannot be explained simply by the activation of retinal receptors. It was these anomalies in particular that got Beau Lotto and me thinking about how color perception might be explained in empirical terms.

Several first-rate psychologists and psychophysicists had studied color contrast and constancy in detail in the early decades of the twentieth century.

But the person whose name had become most closely associated with attempts to rationalize these anomalies was Edwin Land. Land was a Harvard College dropout whose genius had already been established by his many contributions to photography (the invention of the "instant camera" among them). By the late 1950s, he had amassed a substantial fortune from the success of the Polaroid Corporation that he had founded in 1937. Intrigued by some of the mysteries of color vision he had to contend with in color photography, Land took on the challenge of color contrast and constancy and in so doing stimulated a revival of interest in these phenomena. He did so primarily through a series of demonstrations that he presented with considerable fanfare ( Figure 9.3). The best known of these used a collage of physically different papers illuminated by three independently controlled light sources that provided long-, middle-, and short-wavelength light, respectively. (Because the collages resemble the work of Dutch artist Piet Mondrian, such stimuli are often referred to as "Land Mondrians.") Land first adjusted each of the three light sources to some value and then determined the spectral return from one of the surfaces in the array of papers (such as a surface that looked yellowish to observers when it was illuminated by all three lights). Under exactly the same illumination, he showed that another patch in the collage (such as one that looked reddish) provided a substantially different spectrum reaching the eye, as would be expected from the different physical properties of the two papers. Land then readjusted the three illuminators so that the "yellowish" paper now provided exactly the same spectral return to observers as the spectrum that had originally come from the "reddish" paper. Common sense suggests that the "yellowish" paper in the collage should now have looked like the "reddish" paper in the previous condition of illumination. However, the yellowish patch of paper continued to look more or less yellow, and the reddish patch (which was also returning a different spectrum to the eye under the new conditions) continued to look more or less red.

Figure 9.2 A typical color contrast stimulus. The two central squares are returning identical spectra to the eye and are perceived as the same color when presented on the same background, as shown in the key. However, the target in a reddish surround looks yellowish, and it looks reddish in a yellowish surround. (Again, don't be misled into thinking that the key is revealing the "real colors" of the patches; every color perception is a product of the brain, not something that exists in reality.) (After Purves and Lotto, 2003)

Figure 9.3 Land's demonstration of color constancy. A) The appearance of a collage when illuminated by a particular mixture of long-, medium-, and short-wavelength light. Because patches 1 and 2 (see the key) have different physical properties, they return different spectral stimuli to the eye and, as expected, look differently colored. B) The same collage as it would have appeared after readjusting the three sources so that the spectrum returned from patch 2 is now the same spectrum initially returned from patch 1. Despite the drastic change in the spectral returns from the patches, patch 2 continues to look reddish and patch 1 continues to look yellowish. (After Purves and Lotto, 2003)

Figure 9.3 Land's demonstration of color constancy. A) The appearance of a collage when illuminated by a particular mixture of long-, medium-, and short-wavelength light. Because patches 1 and 2 (see the key) have different physical properties, they return different spectral stimuli to the eye and, as expected, look differently colored. B) The same collage as it would have appeared after readjusting the three sources so that the spectrum returned from patch 2 is now the same spectrum initially returned from patch 1. Despite the drastic change in the spectral returns from the patches, patch 2 continues to look reddish and patch 1 continues to look yellowish. (After Purves and Lotto, 2003)

Was this article helpful?

0 0

Post a comment