STIMULUS SPECIFICATION

A visual stimulus has its beginning as a retinal image and exists as a function of both space and time. One of the core questions in vision research is “What is the best way to specify the visual stimulus?” Usually “best” means the stimulus measure that has the simplest relationship with the performance of some visual task and one that accounts for as much of the variance in performance as possible.

Two broad approaches to answering this question have been taken. Historically, specifications of the visual stimulus have been based on the distribution of luminance across a two-dimensional plane whose coordinates are expressed in degrees of visual angle. Spatial measures derived from the luminance specification range from contrast with the background to amount of contour and number of line and edge features in the stimulus. The empirical basis of visual science until the 1960s was based on relationships between performance on one hand (detection, discrimination, recognition, and identification) and some expressed characteristic of the stimulus on the other (contrast, mean luminance, or angular size). Often, in order to give a simpler relationship between a stimulus specification and performance, the stimulus specification is subjected to a mathematical transformation. The most widely used stimulus transform is the logarithm; for example, the logarithm of stimulus luminance or contrast often has a linear relationship with the z-score of percentage correct in the psychometric function.

The second approach to stimulus specification developed in the past 15 years and is based on a rather complicated mathematical transformation of the luminance distribution of the stimulus: the two-dimensional Fourier transform. The basis of this transform is Fourier's theorem, and in this application the theorem states that any two-dimensional image can be expressed as a harmonic series of sinusoidal grating components of the appropriate spatial frequency, amplitude, phase, and orientation. Each image has a unique series of sinusoidal components that, when added together, will recreate the original image.

In general, the Fourier transform of real numbers (e.g., luminance values in space) is composed of complex numbers, having a real and an imaginary component. These complex numbers are more usually represented as having an amplitude and a phase component. When the two-dimensional luminance plane of a visual stimulus is subjected to Fourier transformation, two planes are therefore created. In both planes the coordinate axes are measured in spatial frequency (cycles per degree), not spatial distance (in degrees). The first plane contains the amplitude information as a function of spatial frequency and is called the amplitude spectrum of the stimulus. The second plane contains phase information as a function of spatial frequency and is called the phase spectrum of the stimulus. Each point in the spatial frequency space represents a sinusoidal grating, of a particular spatial frequency, orientation, contrast, and phase. The Fourier transform is reversible: given the amplitude and phase spectrum of a stimulus, it is possible to reconstruct the original spatial stimulus by means of the inverse Fourier transform. Thus no information is lost or gained in going from the spatial to the spatial frequency specification of a stimulus. Two examples of two-dimensional Fourier transforms are shown in Figure 21. In the upper part of the figure are shown the luminance distributions of the letters “B” and “H” (white letters on a black background. Distance above the plane represents luminance. Below the spatial representation of the letters are the two-dimensional amplitude spectra of the letters, calculated by means of the fast Fourier transform. In these spectra, zero spatial frequency is in the center of each plane, with spatial frequency increasing outward from the center in all directions. Contrast is represented by distance above the plane.

FIGURE 21

Spatial and spatial frequency representations of the letters B and H. Top row: The luminance distribution over space--for ease of viewing, they are shown as white letters against a dark background. Second row: The two-dimensional spatial frequency amplitude (more...)

LINEAR SYSTEMS ANALYSIS

One advantage of describing stimuli in the spatial frequency domain rather then in the space domain is that principles of linear systems analysis may be applied. The basic tenet is that if the response of a linear system is known for each of a series of elementary signals, then the response of the system for any stimulus, no matter how complex, can be predicted. In the case of Fourier analysis, the elementary signals are sinusoidal gratings. We can measure how a system responds when presented with individual sinusoidal gratings. Typically, it will respond better to some frequencies (usually low frequencies) and respond progressively less as frequency is increased. The response is measured by how much of the grating modulation present at the input is transferred to the output. This transfer ratio is measured as a function of spatial frequency and is called the modulation transfer function (MTF). The MTF of a system predicts how any stimulus will be transferred through the system because the stimulus can be described as a series of simple sinusoidal gratings, each of which is transferred with some specific transfer ratio.

There are many assumptions necessary to apply linear systems analysis to the human visual system, many of them not valid. Nevertheless, often the consequences of violating these assumptions are not serious and allow a first-order approximation to predicting how the visual system will respond to a stimulus. Since the final response of the visual system is a subjective experience, the MTF of the system cannot be measured directly. The contrast sensitivity function can be used as an approximation to the MTF. A two-dimensional contrast sensitivity function is shown in the third row of Figure 21. It is typically shaped like a volcano: we are more sensitive to intermediate spatial frequencies than to either lower or higher ones. The amplitude spectra of the letters B and H after passing though a system having a MTF shaped like the human contrast sensitivity function are shown in the fourth row of Figure 21. Notice how much high spatial frequency information is removed from the spectra as a consequence of this filtering.

The basis for the use of sinusoidal gratings as test stimuli in the measurement of the contrast sensitivity function is rooted in the desire to apply linear systems analysis in order to understand the functioning of the visual system. Since the elementary signals of Fourier analysis are sinusoidal gratings, they are the stimuli of choice, since it is necessary to know how the visual system responds to these elementary signals if predictions concerning complex stimuli are to be made. Much controversy still exists about the application of Fourier analysis to human vision, but this controversy is largely theoretical in nature. If the visual system were a linear system, than we could predict the appearance of any visual stimuli simply by filtering its two-dimensional Fourier spectrum with a filter shaped like the contrast sensitivity function. Indeed a few investigators have sought to show what the world looks like to a person with amblyopia (Lundh et al., 1981) or other ocular pathology (Ginsburg, 1984a).

GABOR FUNCTIONS

There are sets of elementary signals other than sinusoidal gratings that can be used to measure the sensitivity of the visual system for the purpose of linear systems analysis. One such stimulus is a sinusoidal grating that has been multiplied by a Gaussian function. Several such stimuli are shown in Figure 22. These stimuli are being called Gabor functions, after Dennis Gabor, who in 1946 proposed that they could be used as a set of elementary signals for linear systems analysis (Gabor, 1946). Gabor showed that there is a trade-off between localizing a stimulus in space and localizing it in frequency. For example, a point in space is perfectly localized in space but completely unlocalized in frequency because its frequency spectrum contains all frequencies. An infinitely large sinusoidal grating is perfectly localized in frequency, because it contains only one frequency, but it is completely unlocalized in space because it extends to infinity. Gabor proved that the stimuli shown in Figure 23 maximize the joint localization in both space and frequency simultaneously.

FIGURE 22

Sinusoidal gratings that have been windowed by a Gaussian function, called Gabor signals or Gabor functions. These stimuli are optimally localized both in space and in spatial frequency.

Marcelja (1980) first suggested that cells in the visual cortex have receptive field sensitivity profiles that are of the form of Gabor functions, and further electrophysiological measurements in the visual cortex of monkies support this idea (Kulikowski et al., 1982; Pollen and Ronner, 1982; Pollen et al., 1984). Psychophysical evidence suggests that the human visual system may also contain mechanisms having characteristics of Gabor functions (Daugman, 1980; MacKay, 1981; Watson et al., 1983; Pollen et al., 1984). These developments may have consequences for the way in which the human contrast sensitivity function is measured, but it is too early to know with any certainty what they are.