• acquire knowledge about ourselves and the world around us;
• store this knowledge in memory so that it will be available for use later;
• integrate this new information with prior knowledge;
• use knowledge to plan and execute actions that will cope with environmental demands and achieve personal goals; and
• use language as a tool for thinking, problem-solving, and communication with others

• Sensation detects the presence of an object or event in the environment. The function of our sensory apparatus is to answer such questions as "Has there been a change in the environment?";"Is there something out there?"; and if so,"How strong is it?".
• Perception forms a mental representation of that object or event. The function of our perceptual apparatus is to answer such questions as "What is out there?","Where is it?", and "What is it doing?".

• The distal stimulus is the object of perception itself -- the tree, or rock, or ocean, or car that exists in the world outside the mind.
• The proximal stimulus consists of all the physical energies emanating from the distal stimulus, and which fall on the sensory receptor organs associated with our eyes, ears, etc. The proximal stimulus might be a pattern of light waves reflected from the tree, or a pattern of sound waves generated by the motion of its branches and leaves as it sways in the breeze, or some other kind of physical energy.

• At the same time, there are possible exceptions, where interoception does appear to give rise to conscious sensation -- for example, when you feel hungry or thirsty, chilly or feverish.  Hunger has to do with blood-sugar levels, and thirst has to do with cell fluids, and it's not clear that people really have sensory awareness of these physiological parameters. 
  
• But people do feel something when they're hungry or thirsty -- though maybe these sensations arise from contractions in the stomach or dryness in the mouth -- which wouldn't exactly be interoception, at least in Sherrington's terms. 
  
• People with hypoglycemia often claim that they can tell when their blood-sugar levels have gone down.  But it's not clear that they're sensing blood-sugar levels directly.  Instead, they may be making an inference about their blood-sugar levels based on other symptoms, such as dysphoria or dizziness.
• It's been claimed that difficulties with interoception are risk factors for eating disorders (such as anorexia, bulimia, or binge eating) and body dysmorphic disorder.  Certainly, patients with eating disorder engage in eating behaviors (including fasting) that are discordant with their actual internal states (see "Inside the Wrong Body" by Carrie Arnold, Scientific American Mind, May-June 2012). (More about these disorders in the lectures on Psychopathology and Psychotherapy.)
  
• Interoceptive skill is sometimes measured with the "heartbeat test" developed by Hugo Critchley and his colleagues: scores on this test are highly correlated with laboratory measures of interoceptive awareness.
• Sit in a comfortable chair, take a few deep breaths; start a stopwatch, and count your heartbeats for 1 minutes (without taking your pulse!).  This is your estimated pulse.
  
• Then take your pulse by putting your fingers on your wrist or neck and counting for one minute.  Then rest for a minute and do it again.  Take the average, to get your average pulse.
  
• Subtract your average pulse from your estimated pulse.
• Divide the absolute value of the difference by your average pulse to get a decimal.
• Subtract this decimal from1.
• A score of 0.80 or higher indicates "very good" interoceptive skill.
• A score of 0.60 to 0.79 indicates "moderately good" skill.
• A score or 0.59 or less indicates "poor" interoception.
• Cognitive-behavioral therapists often use biofeedback to help people learn how to control their internal physiological states.  The whole point of biofeedback is that people can't sense these states directly, and need special monitors to gain this awareness indirectly.  (More on biofeedback in the lectures on Psychopathology and Psychotherapy.)

• A paper by Martha McClintock (Nature, 1971) reported that roommates and close friends at Wellesley College, then as now an all-women's residential college, tended to synchronize their menstrual cycles (she obtained the same effect among cloistered Roman Catholic nuns).
• A paper by Noam Sobel (Science, 2011) reported that men show diminished sexual arousal and desire when exposed to "emotional tears" produced by women crying in response to sad stimuli.

• free nerve endings buried in the epidermis of the skin;
• "perifollicular"or "basket" endings wrapped around the base of hair follicles;
• Merkle's disks are sensitive to pressure and rough texture;
• Meissner's corpuscles are sensitive to light touch, vibration, and stimulus texture; and
• Pacinian corpuscles are sensitive to vibration and deep pressure;
  
• Ruffini's corpuscles are sensitive to pressure, vibration, and stretching of the skin.

• Fast pain -- sharp or prickling, with a rapid onset and rapid offset -- appears to be mediated by A-delta fibers in the spinal (and some cranial) nerves.
• Slow pain -- aching, throbbing, or burning, with a slow onset and slow offset -- appears to be mediated by C-fibers in the same nerves.

Fast pain sensations then travel up the spinal cord along the neospinothalamic tract, while slow pain sensations then travel up the spinal cord along the paleospinothalamic tract.

Fast pain impulses pass through the thalamus to the somatosensory cortex, while their slow pain counterparts terminate in the brainstem.

Pain sensations usually result from injury to some tissue, like the skin.  But chronic pain can persist even after the injury has healed.  In many cases of chronic pain, neurons in the "pain pathway" become abnormally excitable, and discharge impulses toward the brain even though they're not receiving any input from nociceptors.

Here's a good diagram showing our best understanding of the physiology of pain (from "A World of Pain", by Yudhijit Bhattacharjee, National Geographic, 01/2020).  In this approach, pain sensations arise in tactile receptors that are especially sensitive to points or sharp edges, heat, and the like.  In a "bottom-up" process, these signals are carried over afferent neurons which contain both A-delta and C fibers to ascending tracts in the spinal cord, through the thalamus, and then to the somatosensory cortex, which identifies the location and intensity of the pain.  The thalamus  (which, remember, serves as a sensory relay station) also sends the signals to other brain centers, such as the anterior cingulate cortex, anterior insular cortex, which mediate the individual's emotional response to the pain.

The same diagram shows the gate-control mechanism for pain control.  Following the onset of pain, a "op-down" process is initiated in the prefrontal cortex to modulate the pain.  These descending signals contact the ascending pain signals in the dorsal horn and periaqueductal gray of the spinal cord, effectively reducing the intensity of the ascending pain signals. 

Usually, we think of pain as a skin sense, but pain is more complicated than this. Almost any stimulus will produce pain if it is intense enough: sound, light, heat, or cold are good examples (it is not clear if intense tastes and odors are actually painful, no matter how unpleasant they might be). Moreover, pain is also produced by muscle fatigue, and by the introduction of salt and other chemicals into wounds and other openings in the skin. So setting aside cutaneous pain, in some sense there is no unique proximal stimulus for pain. And because pain can be experienced in (almost) all sensory modalities, in a sense there is are no unique receptor organs for pain, either -- and no specific sensory tract, and no specific projection area.

The situation is complicated further by the fact that, regardless of origin, pain appears to have two broad components:

In the final analysis, the experience of pain is substantially determined by the meaning of the stimulus. In doctors' offices, people flinch when there isn't much pain. The responses of injured children differ markedly depending on whether an audience is present. Placebos provide genuine relief of pain. Soldiers experienced different levels of pain from objectively similar wounds in World War II, Korea, and Vietnam. And many religious rituals in tribal cultures (e.g., fertility, puberty) would appear objectively painful, but elicit no signs of pain from the participants.

The bottom line is that pain is often more a problem of perception than of sensation. The experience of pain depends not so much on the transduction of stimulus energies, but rather of higher mental processes involving perception, memory, and thought.

Review of the Skin Senses

So just to review how the skin senses work. Mechanical pressure, vibration, or tissue inflammation stimulate specialized sensory receptors in the skin. This generates neural impulses which travel over the afferent branch of the spinal nerves to the spinal cord. There they may generate an involuntary spinal reflex in response to the stimulus. Then the sensory impulses travel up the ascending tract of the spinal cord to the somatosensory projection area in the parietal lobe. There they might generate a voluntary response to the stimulus.

Kinesthesis

The sense of movement and position is generated by activity in the skeletal musculature -- the contraction of tendons, and stretching of muscles (and of the skin), and the movement of joints.

These activities stimulate nerve endings that are embedded in these body parts, including

These impulses, too, are carried over the afferent tracts of the spinal nerves, and up the afferent tract of the spinal cord as well as over some of the cranial nerves.

And finally to the somatosensory projection area of the parietal lobe.

Equilibrium

Equilibrium, the sense of balance, begins with gravitational force which pulls on otoliths, tiny crystals suspended in the semicircular canals, and the saccule and utricle of the vestibular sac, both structures located in the inner ear. The semicircular canals are arranged at approximate right angles to each other, and detect rotation of the head.  The saccule and utricule detect the position of the head with respect to  gravity.

These structures contain hair cells, which are stimulated by the crystals, generating neural impulses in much the same manner as the cochlea does for audition.

The neural impulses are then carried over the vestibular branch of the vestibulocochlear nerve (VIII).

Unlike kinesthesis and the skin senses, neural impulses relating to equilibrium and change of motion carried over the vestibular division of cranial nerve VIII project to the cerebellum.

The Doctrine of Specific Nerve Energies

To repeat, in the abstract, each modality of sensation is characterized by four factors:

• the proximal stimulus (or type of physical energy falling on a receptor organ);
• the receptor organ which is stimulated (ordinarily, prepared to respond to a particular proximal stimulus);
• the sensory tract leading from the sensory surfaces to the brain, including the afferent tract of the spinal nerves and the afferent tract of the spinal cord, as well as certain cranial nerves; and
• the sensory projection area in the cortex, a particular area of the brain which is the final destination of afferent impulses arising from the sensory receptors and transmitted along the sensory tracts.

At first glance, it seems that each sensory receptor organ is specific to a different kind of proximal stimulus -- which would mean that the various sensory modalities are actually defined by particular kinds of proximal stimuli.

This would be a great, and simple, system. Unfortunately, it turns out that the modality of sensation is not uniquely associated with any particular proximal stimulus or receptor organ.

For example, other stimuli can lead to a particular sensation. If you close your eyes and press your finger lightly on your eyelid, you will experience the sensation of touch stemming from pressure that deforms the surface of the skin. But you will also experience the sensation of vision, in terms of various lights and colors. Thus, a particular proximal stimulus (pressure) somehow stimulates the receptors for vision as well as those for touch. Similarly, if you box someone's ears (Please don't do this, just take my word for it!) the victim will experience touch (to put it mildly), but also a ringing of the ears.

Electrical stimulation of various sensory receptors can also give rise to sensations -- which is how cochlear implants (and, soon, retinal implants) work.

Proof that the modality of sensation can't be defined in terms of either the proximal stimulus or the sensory receptor comes from electrical stimulation of the afferent nerves leading from the receptor organ to the central nervous system, or of the sensory projection areas in the cerebral cortex. Either case produces sensory experience in some modality, vision or audition or whatever, even though there has been no proximal stimulus in the usual sense nor any activity in any sensory receptor per se.

Considerations such as these led the German physiologist Johannes Muller, in 1826, to formulate the Doctrine of Specific Nerve Energies.

The same cause, such as electricity, can simultaneously affect all sensory organs, since they are all sensitive to it; and yet, every sensory nerve reacts to it differently; one nerve perceives it as light, another hears its sound, another one smells it; another tastes the electricity, and another one feels it as pain and shock. One nerve perceives a luminous picture through mechanical irritation, another one hears it as buzzing, another one senses it as pain. . . He who feels compelled to consider the consequences of these facts cannot but realize that the specific sensibility of nerves for certain impressions is not enough, since all nerves are sensitive to the same cause but react to the same cause in different ways. . . (S)ensation is not the conduction of a quality or state of external bodies to consciousness, but the conduction of a quality or state of our nerves to consciousness, excited by an external cause.

According to Muller, the modality of sensation is not determined by the proximal stimulus alone, but rather by the stimulation of particular nerves. These nerves are ordinarily responsive to some specific proximal stimulus, but they can also respond to others. You can have a sensory experience without a proximal stimulus, so long as some particular part of the nervous system is activated.

Muller had the right idea, but the DSNE must be restated in light of subsequent evidence.

The implication is that the modality of sensation is not determined by the proximal stimulus, nor by the sensory receptors or even by the sensory nerves (as Muller thought), but by the sensory projection area where the neural impulses carried by the sensory nerves end up. It doesn't matter where the impulse starts out -- it matters where it ends up. Theoretically, if we could connect up the auditory nerve to the visual projection area, we would see sounds rather than hear them.

And, in fact, not just in theory: Roger Sperry, and American neurophysiologist, studied certain species of amphibians, whose optical pathways actually cross completely. Instead of the usual situation, where the left visual field of each eye projects to the right hemisphere, and vice-versa, the whole left eye projects to the right hemisphere (and vice-versa). Sperry observed that frogs would strike at small, bug-like objects presented to their left eyes. He then severed the optic nerves and regrew them so that the right eye connected to the right hemisphere, and vice-versa. After the regeneration, the animals actually reversed their responses -- they behaved as though the stimuli were on their right side, for example, when they were actually on the left. (Sperry won the Nobel Prize, in part, for these experiments).

So, in the final analysis, it's the visual projection area that gives us the sensation of seeing -- not light waves, or the rods and cones, nor the optic nerve.And it's the auditory projection area that gives us the sensation of hearing -- not sound waves, or the hair cells in the cochlea, nor the cochlear nerve.

On this assumption, here's how nine sensory modalities might be defined:

Difficult Modalities

The definition of a sensory modality is very clear for vision and audition, and pretty clear for olfaction and gustation. But other modalities of sensation are a little more muddled. 

The Skin Senses

As should be clear by comparing the pathways for touch, temperature, and kinesthesis, the skin senses are hopelessly complex. At least seven different nerve endings are found in the skin, but there is little isomorphism (one-to-one relation) between the type of receptor, and the corresponding sensory experience when that receptor is stimulated electrically. To make things worse, neural impulses from all six receptor types are transmitted along the same afferent tracts to the spinal cord and up to the brain, and all project to the same somatosensory cortex in the parietal lobe. So the puzzle is: how are the skin senses kept separate? There is a prize for the person who comes up with the best answer (it's awarded in Stockholm).

Pain is another problematic modality, in large part because the experience of pain depends so much on the meaning of the pain stimulus. As far as sensory pain is concerned, the critical projection area is in S1, the primary somatosensory cortex. But then again, even very bright lights and very loud sounds can be painful, and these stimuli register in the visual and auditory areas, respectively. So the projection area for sensory pain will correspond to the modality in which the sensation occurs. But suffering, which transcends any particular sensory modality, seems to be processes in the anterior cingulate gyrus and other structures in the medial frontal lobe. The point here is that pain is both a sensation and a perception. 

Maybe what's needed is a different approach entirely.  A different take on the skin senses was proposed by Melzack and Wall (1962), who noted that there is not, in fact,a one-to-one correspondence between the physiological and experiential aspects of somesthesis.  They criticized the class definition of the sensory modalities (or, at least, the skin senses) in terms of specific receptor organs sensitive to specific physical stimuli, which generated neural impulses which traveled over specific afferent pathways to specific projection areas in the brain. Following Hebb (1955), and Skinner (1938), they criticized this conceptual nervous system, which envisioned a kind of direct neural channel from the skin to the brain.  (Another example of a conceptual nervous system is the reflex arc introduced in the lectures on the Biological Bases of Mind and Behavior: obviously, things are more complicated than circuits of three neurons!).  They acknowledged that there are many different types of receptors embedded in the skin, but they argued that these receptors do not correspond to any particular modality (such as the tactile or thermal sense), or any quality within a modality (such as warmth or cold). 

Melzack and Wall made the same argument with respect to the peripheral nerve fibers that connect the receptor organs to the central nervous system.  The original idea was that the different modalities (or qualities) were associated with fibers of different size:

But careful investigation determined that these various fibers do not "monopolize" a particular modality or quality.  For example, the C fibers also carry impulses generated by pressure, light touch, and itch.  And the same has to be said of the central pathways that carry somesthetic impulses up the spinal cord to the brain.  So, contrary to earlier speculation, the dorsal spinal tract is not uniquely associated with touch, and the lateral spinal tract is not associated uniquely with pain.

In an attempt to resolve this situation, Melzack and Wall proposed that the modality of skin sensation is determined by the specific temporal and spatial pattern of neural impulses generated by a whole host sensory receptors embedded in the skin.  Each receptor organ is sensitive to a particular range of stimuli.  Some receptors may be sensitive to warmth, but also to light touch.  Other receptors might be sensitive to touch and the chemical changes that result from injury to the skin.  And so on.  As they put it, "Every discriminably different somesthetic perception is produced by a unique pattern of nerve impulses (p. 350). 

Melzack and Wall (1965) elaborated their theory in their gate control theory of pain.  Again, they examined the failures of specificity theory, and concluded that "pain" isn't transduced by any specific receptor, nor carried along any particular afferent tract.  As they noted:

The stimulation of a single tooth results in the eventual activation of now less than five distinct brain-stem pathways.  Two of these pathways project to cortical somatosensory areas I and II, while the remainder activate the thalamic reticular formation and the limbic system, so that the input has access to neural systems involved in affective as well as sensory activities (p. 976).

Instead, they proposed that pain is produced by neural impulses generated by the free nerve endings and other receptors, which are carried over both the large-diameter, myelinated A-delta fibers and the small-diameter, unmyelinated C fibers to the dorsal horn of the spinal cord.  There they connect to transmission (T) cells that carry the impulses further toward the brain, and inhibitory cells which -- as their name implies -- create a kind of gate through which the pain signals must pass.   

But what about itching?  The story begins with specific itch-inducing irritants, known as pruritogens (pruritus is the medical term for itching).  These chemicals include histamine, chloroqune, and cowhage.  In response to a bug bite, for example, the immune system attacks foreign chemical bodies deposited by the insect.  The immune system then releases histamine, which stimulates specific receptors, causing a sensation of itching.  There are also specific receptors for non-histaminic irritants, which also cause itching.  These are known as mas-related G-protein-coupled receptors (Mrgprs).  What do you do when you feel an itch?  You scratch it, that's what you do.  And scratching damages the epidermis, which stimulates the release of more histamine and other pruritogens, making the situation even worse.  That's why a common treatment for itching are antihistamines.  But histamines don't always solve the problem, because there are other pruritogens that antihistamines don't affect.  There was once a theory that itching was just a mild form of pain -- or maybe a special form of touch sensation.  But the discovery of pruritogens, which don't cause pain, and receptors which are sensitive to pruritogens but not sensitive to pain, seems to suggest that "itching" is a separate tactile modality from touch, temperature, and pain. 

For more on itching, see "The Maddening Sensation of Itch" by Stephani Sutehrland,  Scientific American, May 2016.

Time

And finally, there's time. We speak of people having a "sense of time", but is time really a sensory modality? And if it is, what is the proximal stimulus, and what is the projection area?  Some neuroscientists claim to have identified a projection area for time in the basal ganglia, but there is almost certainly not a single projection area for time, any more than there is one for pain. On the other hand, we know that the body contains a fairly large number of biological clocks, beginning with the one that regulates the diurnal cycle of waking and sleeping. A classic study by Hoagland found that fever patients (actually his wife) consistently underestimate the passage of time, suggesting that increases in body temperature speed up whatever clock resides in our brain. Similarly, time seems to pass quickly when we're excited, but slowly when we're bored. As with pain, our "sense" of time is probably better thought of as a matter of perception. For an engaging early discussion of the problem of time, see Robert Ornstein's On the Experience of Time (1969).

Where It Ends Up

So, although in principle, the modality of sensation is determined by four factors (proximal stimulus, receptor organ, sensory tract, and sensory projection area), in the final analysis the projection area is the most important. If the auditory nerve were connected to the visual cortex, we would see sounds instead of hearing them (we know this because direct electrical stimulation of the visual cortex produces visual sensations).Regardless of where the sensory impulse comes from, what matters is where it goes.

On that principle, maybe Aristotle got it right after all: there is only a single skin sense, with pressure, warmth, cold, and pain being qualities of tactile sensation. But let's not go there.

Nine Ways of Knowing the World ...and More?

So, based on what we now know, there appear to be nine different ways of knowing the world: Aristotle's five -- seeing, hearing, tasting, smelling, and touching; two additional skin senses -- temperature and (cutaneous) pain; and Sherrington's two proprioceptive senses -- kinesthesis and equilibrium.

There may well be others. Recall that Sherrington identified a category of interoception that processes stimulation from internal tissues and organs, such as the viscera and the blood vessels. We'll talk more about interoception in the lecture supplement on Motivation, where we cover basic biological motives like hunger and thirst.

A Magnetic Sense?

Studies of migratory animals, especially birds and fish but also some mammals, suggest that some animals mammals respond to the earth's geomagnetic field. While many migratory animals respond to celestial cues, this magnetic sense may also serve as an aid to navigation.

In fact, there may be two magnetic senses:

If there's a magnetic sense, what's the sense organ? That's still pretty much a mystery.

"Extrasensory" Perception?

Another controversial modality raises the question of extrasensory perception (ESP) -- the acquisition of information without mediation by the sensory system(s). ESP is thought to be manifested by telepathy (thought transference),clairvoyance (perception of objects that are not influencing sensory receptors), and precognition (perception of future events). It also covers action without mediation by effector systems, as manifested by psychokinesis (PK; manipulating an object without touching it).

These phenomena are often associated with the occult, and reflect beliefs in supernatural powers. But they are also of interest to scientists. (1) Research may reveal previously unrecognized sensory modalities. (2) Even if not, the experience of telepathy, clairvoyance, etc. is psychologically interesting, even if claims of ESP are not valid.

In fact, there is very little compelling evidence for either ESP or PK. Most ostensible psychics are in fact professional magicians or illusionists, and the trickery involved in their demonstrations can be detected by their professional peers. Moreover, most laboratory studies of ESP and PK in ordinary people don't effectively prohibit cheating (by either experimenter or subject), and are poorly controlled in other respects.

Even the best ESP/PK research poses numerous empirical problems.

The evidence is not all in, and it is best to keep an open mind, but when one removes outright fraud, poor methodologies, and capitalization on chance, there is very little phenomenon left to explain.

Sensory Integration

So there are nine ways of knowing the world, each associated with its own neural system.  But these nine sensory modalities do not operate independently of each other.  Rather, information from the separate modalities is blended, and the various modalities influence each other.

Perhaps the clearest demonstration of this is known as the McGurk effect (McGurk & MacDonald, 1976; see Rosenblum, 2008)). 

Thus, visual and audio information are blended together to produce a sensation that is quite different from either stream of stimulation.

The opposite phenomenon, in some sense, is known as visual capture.  If you watch two actors conversing on a television set where the sound comes from a single speaker, you still perceive the voices as coming from different people (Thurlow & Jack, 1973).  Ventriloquists use this trick too, which is why visual capture is also called the ventriloquism effect.  In either case, sounds are perceived as coming from a different direction than their actual source; and this auditory effect is created by visual stimulation.

The Doctrine of Modularity sometimes conveys the impression that the brain is like a Swiss Army Knife, with lots of different tools contained in a single case.  And, in some respects, that's true.  There do seem to be different modules in the brain, each specialized for a particular function.  And, in some sense, that is true of the sensory modalities as well.  But just as with the brain, the different pieces -- in this sense, the different sensory modalities -- work together to create a unitary sensory experience

For more on sensory integration, see "A Confederacy of Senses" by Lawrence D. Rosenblum, Scientific American, January 2013.

The Sensory Qualities

Each sensory modality is associated with a unique sensory experience -- e.g., vision, taste, touch, and balance. We now know something about how these experiences are related to proximal stimuli, and how these stimulus energies are transduced into neural impulses by sensory receptors and carried over various sensory tracts to the several projection areas. But transduction is only the first problem of sensation.

Within each modality, there are different sensory qualities.

How are these qualities sensed?

The Psychophysical Principle

The 19th-century psychophysicists began by attributing each quality of sensation to a specific proximal stimulus. And, in fact, their research was guided by the General Psychophysical Principle that each psychological quality of sensation was associated with a particular physical property. And, in fact, psychophysics was generally dedicated to tracing out these psycho-physical correlations.

Here's how the psychophysical principle might work out, with respect to the two most commonly studied sensory modalities, vision and audition.

Visual hue is related to the wavelength of visual stimulation.

Saturation is varied by adding gray to a particular hue. So, adding gray to pure red produces a pinkish sensation.

Auditory pitch is related to the frequency of auditory stimulation.

Note that an interval of an octave doubles the frequency.

Modern orchestras tune to "A-440", second-space A on the treble clef, which corresponds to a frequency of 440 cps.

Timbre is related to the shape, or complexity, of the sound wave.

Based on this simple psychophysical principle, then, we could speculate that the quality of a sensory experience is determined by the physical properties of the stimulus which impinges on the sensory receptors.

But we know that that simple principle doesn't work to determine the modality of sensation. Rather, according to Muller's Doctrine of Specific Nerve Energies, the modality of sensation is determined by the particular neural structures that process the transduced sensory input. It turns out that this is also the case when it comes to the qualities of sensory experience.

The Doctrine of Specific Fiber Energies

In fact, Hermann von Helmholtz, a 19th-century German physiological psychologist, proposed an extension of Muller's Doctrine of Specific Fiber Energies (actually, Helmholtz was Muller's student). Just as each modality has its own associated neural structures, so Helmholtz proposed that each quality is differentiated in a similar manner. There may well be particular psychophysical relationships, just as vision is normally stimulated by light waves and hearing by sound waves. But Helmholtz also argued that there are also separate separate neural structures corresponding to each different quality of sensation, and that these are decisive. As with the modality of sensation, in the final analysis what distinguishes among the various qualities of sensation are the particular neural structures which are activated -- not the proximal stimulus itself.

Helmholtz' proposal stimulated a search for these separate, quality-specific "energies", and the particular receptors that transduce them into neural impulses. But a problem presents itself almost immediately: in most modalities there are literally thousands of discriminably different qualities.

Auditory Pitch

You get a sense of how the doctrine worked with Helmholtz's own early example, auditory pitch.We have already seen that vibration of the tympanic membrane sets up sympathetic vibrations of the basilar membrane against hair cells. Looking through a microscope, he determined that the basilar membrane actually consists of a large number of fibers stretched across a frame, not unlike a harp or a piano.  These fibers vary in length and thickness.  Accordingly, Helmholtz suggested (in his monograph On the Sensations of Tone, 1863) that each frequency causes a different fiber to vibrate against its corresponding hair cell. Complex stimuli set a number of different fibers into vibration, corresponding to the fundamental frequency, overtones, etc. This is now known as the place theory of pitch.

Helmholtz's speculations were supported by research by Georg von Bekesy (1960), who observed that high-frequency tones produced strongest vibrations in portions of the basilar membrane nearest the eardrum, while lower-frequency tones produce the strongest vibrations near the tip of the cochlea. This appeared to confirm that pitch is coded by the place of maximal stimulation, and Bekesy won the 1961 Nobel Prize in Physiology or Medicine.

Helmholtz's theory of pitch perception that has played an important role in theories of musical harmony. The English Novelist George Eliot made use of it in her last novel, Daniel Deronda (1876). See J.M. Picker,Victorian Soundscapes (2003).

However, the pure place theory has problems, not the least of which is that, at low frequencies, the basilar membrane vibrates as a whole, at a frequency roughly corresponding to the frequency of the stimulus.

Accordingly, the prevailing duplex theory of pitch perception employs a number of different principles:

Wait a minute: That's three principles, whereas the term "duplex theory" implies that there should only be two. Note that, if you think about it for a second, that's really only two principles, because the frequency principle and the volley principle are actually the same frequency principle: that the frequency of stimulation is coded by the frequency of the resulting neural impulse (at low frequencies, the neural impulse is generated directly; at higher frequencies, by means of volleys). That, and the place principle, makes up the "duplex".

Color Vision

Later on, Helmholtz applied of the Doctrine of Specific Fiber Energies to in color vision (in his Physiological Optics, which was published serially from 1856-1866, and appeared complete in 1867).

Consider all the different color patches that may be found at a paint store: as of 2006, Pantone, the Dutch paint company favored by high-market interior designers, offers a color wheel with 3,039 specific hues, including about 300 shades of blue. Or, for that matter, the "millions of colors" that can be produced on a high-quality computer screen. There are probably more colors than can actually be resolved by the human eye (although the human eye is pretty good at this, being able to distinguish among 7 million different shades of color produced by unique combinations of hue, brightness, and saturation). Brightness has to do with how light or dark a color is; saturation has to do with the purity of the color.

Just to illustrate the infinite variety of colors that can be produced by combining hue, variety, and saturation, every year Pantone releases a "Color of the Year" (a competing "Color of the Year" is advertised by the Color Marketing Group: interestingly, theirs doesn't match up with Pantone's!). Here's an interesting exercise.  Dig out your favorite photo-processing software (I use Photoshop), go to the color swatches, and try to reproduce these colors by blending red, green, and blue -- just like your eye does.  Make some adjustments, as an experiment, and see what happens to the color.
For 2010, it was "Turquoise".  In the RGB system, "Turquoise" is obtained with R=68, G=184, and B=172.
For 2011, it was "Honeysuckle". In the RGB system, "Honeysuckle" is obtained with R=214, G=80, and B=118.
For 2012, it was "Tangerine Tango". In the RGB system, "Tangerine Tango" is obtained with R=221, G=65, and B=36.
For 2013, it was "Emerald". In the RGB system, "Emerald" is obtained with R=0, G=152, and B=116.
For 2014, it was "Radiant Orchid". In the RGB system, "Radiant Orchid" is obtained with R=177, G=99, and B=163.
For 2015, it was "Marsala". In the RGB system, "Marsala" is obtained with R=150, G=79, and B=276.
For 2016, the Color of the Year was actually a pairing of two colors, "Rose Quartz" and "Serenity".  In the RGB system, "Rose Quartz" is obtained with R=247, G=202, and B=201. In the RGB system, "Serenity" is obtained with R=146, G=168, and B=209.
For 2017, the Color of the Year was "Greenery", which some observers noted is the color of the marijuana leaf! In the RGB system, "Greenery" is obtained with R=136, G=176, and B=75.
For 2018, the Color of the Year was "Ultra Violet".  Leatrice Eiseman, Executive Director of the Pantone Color Institute, said that ultra violet is "also the most complex of all colors because it takes two shades that are seemingly diametrically opposed -- blue and red -- and brings them together to create something new" ("The Fture is... Purple" by Vanessa Friedman, New York Times, 12/07/2017). In the RGB system, "Ultra Violet" is obtained with R=95, G=75, and B=139.
For 2019, the Color of the Year was "Living Coral", which some critics called "the color of grandmothers".  Pantone, however, called it "a big hug" -- one of the "colors that bring nourishment and the comfort and familiarity that make us feel good. It's not too heavy. We want to play. We want to be uplifted." In the RGB system, "Living Coral" is obtained with R=221, G=77, and B=67.
For 2020, the Color of the Year was "Classic Blue", which seems to have been intended as a kind of antidote to the anxiety-laden 2019 -- or perhaps a prophylactic against an anxiety-laden 2020 (or both).  Noted an article in the New York Times ("Pantone Declares Another Year of Blue" by Jessica Testa, 12/05/2019): Classic Blue is the color of blueberries, a Pepsi can and the sk when it's "that beautiful color at the end of the day," said Leatrice Eiseman, the executive director of the Pantone Color Institute....  In choosing Classic Blue, the organization said it first examined what was going on in the world. "We're living in this time now where things seem to be, around the world, a little bit, I don't want to use the word unstable, but let's just say a little shaky," said Laurie Pressman, the vice president of the Pantone Color Institute.  "Nothing is absolutely certain from one moment to the next".... Classic Blue "provides a refuge,, according to Pantone, fulfilling a "desire for dependable, stable foundation".  Classic Blue is "nonaggressive," "easily relatable" and "honest" In the RGB system, "Classic Blue" is obtained with R=15, G=76, and B=129.
In 2021, in the midst of the worldwide Covid-19 pandemic, Pantone named two colors as the Color of the Year for 2021:  Ultimate Gray and Illuminating, a shade of yellow.  In its press release, the company described the pair as "a marriage of color conveying a message of strength and hopefulness that is both enduring and uplifting" -- pretty much what everyone needed right then. In the RGB system, "Illuminating" is obtained with R=245, G=223, and B=77. "Ultimate Gray is obtained with R=147, G=149, and B=151.
In 2022, with a Covid vaccine available, and the world seeing the light at the end of the pandemic tunnel, Pantone introduced Veri Peri (as in "Periwinkle"), described as "a vibrant colour that displays a spritely, joyous attitude and dynamic presence". In the RGB system, Veri Periwinkle is obtained with R=102, G=103, and B=171.
The Color of the Year for 2023 was Viva Magenta, which "vibrates with vim and vigor" and is "expressive of a new signal of strength".  Magenta is a shade of purpose, and purple is produced by a balance between red and blue.  In the RGB system, Viva Magenta is obtained with R=255, G=0, and B=255.  To see what a difference brightness makes, compare with R=100, G-0, and B-100.  Or compare with "Barbie Magenta" below.
The Color of the Year for 2024 was Peach Fuzz.  In discussing their choice, Pantone wrote that "In seeking a hue that echoes our innate yearning for closeness and connection, we chose a color radiant with warmth and modern elegance. A shade that resonates with compassion, offers a tactile embrace, and effortlessly bridges the youthful with the timeless." In the RGB system, Peach Fuzz is obtained with R=190, G=52, and B=85.
A 2012 consumer survey identified opaque couché, a combination of dark brown and olive green (technically, Pantone 448C; in the RGB system, it's coded as74/65/42), as the "ugliest color in the world" (frankly, it looks like...).  In 2016, it was used on cigarette packages in Australia and the United Kingdom, in an attempt to deter people from smoking.  Coming soon, maybe, to a cigarette pack near you?
It's not the Color of the Year, at least not yet, but Olga Alexopoulou, a Greek artist living in Istanbul, collaborated with researchers in the lab of UCB provost Paul Alivisatos, a chemist, to make the perfect “blue hour” blue -- the color of the sky at twilight.  To do so, they employed nanotechnology -- binding "quantum dots" to pigment to give it a special fluorescent quality.  The  pigment, known as Quantum Blue, was announced in 2020; other pigments, such as Quantum Red, Green, and Yellow are sure to follow (patent pending, of course; as Pantone has demonstrated, there is serious money to be made in pigments).  (Image on the left, a bottle of Quantum Blue; on the right, a seascape by Alexopoulou employing the pigment, both published in the Huffington Post, 09/07/2018.)
Barbie, the iconic toy character, loves pink.  Really loves pink.  Really loves really pink stuff.  So she's got her own Pantone color, 219C, technically a shade of magenta-pink.  Compare with Viva Magenta, above.  For more details, see "Barbie: A Visual Dictionary" by Louis Lucero II (New York Times, 07/16/2023), from which the image to the right is taken. In the RGB system, "Barbie's magenta" is obtained with R = 218, G = 25, and B = 132.

Another repository of color is the Forbes Collection, housed in the Straus Center for Conservation and Technical Studies at Harvard.  The Collection contains an incredible variety of pigments housed in various sorts and sizes of tubes and jars housed in glass cabinets and drawers arranged according to the Color Circle.  Most artists these days use synthetic pigments purchased in tubes, but before recent times they ground their own pigments from raw materials.  The Forbes Collection is particularly useful for the conservation and restoration of historic art made from these materials.  Included in the collection is "Vantablack", the blackest black known, which absorbs fully 99.96% of light waves -- the Black Hole of pigments (there's also  "whitest white", which reflects 98.1% of the sunlight that hits it, compared to 10-20% for conventional white paint).  For more on the  Forbes Collection, including the role it  played in the restoration of Harvard's own Rothko Murals, see "Blue As Can Be" by Simon  Schama, New Yorker, 09/03/2018. 

So where does color, or more properly hue, come from?

In the 19th century, Thomas Young discovered that visual hue was related to the wavelength of the light stimulus, as described above (Young also translated the Rosetta Stone). Shorter wavelengths are associated with blue and green, longer wavelengths with red and violet. So, one solution to the problem of color vision would be to have a different receptor for each particular wavelength -- somewhat analogous to Helmholtz's original place theory of pitch perception. But that won't work, for the simple reason that there are too many wavelengths! Do the math: In order for every 1 nm of wavelength of light in the visible spectrum to stimulate a different receptor, there would have to be (at least) 400 different receptors; to make things worse, since we can see any color at any point in space, all 400 receptors would have to be located at each point on the retina. The retina is just not big enough.

Rather, there must be some set of elementary colors, with specific receptors sensitive to each; and the remaining colors must represent blends of these basic elements. What might these basic colors be?

One clue comes from the fact that, as Sir Isaac Newton discovered in the 17th century, the spectrum of visible light consists of seven primary colors:

Red, Orange, Yellow, Green, Blue, Indigo, and Violet

-- the acronym "ROY G BIV" you learned in elementary school. When you pass white light through a prism, it breaks up into these colors. Passing sunlight through raindrops has the same effect, yielding rainbows; in fact, the English poet John Keats accused Newton of "unweaving the rainbow".

Actually, we now recognize only six primary colors. Newton wanted there to be seven, so that the colors of the rainbow would parallel the notes of the diatonic musical scale (C-D-E-F-G-A-B), and the number of planets (he didn't know about Uranus or Pluto) so he made a somewhat arbitrary distinction between indigo and violet. These days, color scientists tend to speak only of violet, or perhaps purple. Long before Newton, Aristotle asserted that there were just four primary colors, each one related to the four elements of Greek physics -- earth, air, water, and fire.

If we mix all the colors together, we get white again; if there is an absence of light, we get black.

Thus, sticking with Newton's count for purposes of exposition, perhaps there are seven color receptors, each sensitive to a particular segment of the visible spectrum corresponding to pure versions of the seven primary colors. Stimulation of one of these receptors would yield one of the primary colors. Stimulation of two or more would yield various blends of color. Stimulation of all at once would produce white, while the absence of stimulation would produce black.

Actually, anatomical evidence indicates that there are two kinds of receptors in the retina:

So, maybe there are seven different cones, each corresponding to one of the seven primary colors. Maybe, but other considerations narrow down the number of cones that are needed to produce color vision.

For example, it appears that four colors are psychological primaries: red, yellow, green, and blue.

Orange and violet are usually perceived as mixtures (i.e., people can identify some combination of red, yellow, green, and blue in them).

In fact, these four colors are also physical primaries, meaning that any hue can be produced as a combination of them.

When subjects are asked to make similarity judgments about colors, the results can be presented in a color circle in which similar colors are adjacent to each other, and complementary colors (those which when mixed additively produce gray) are opposite to each other. Thus, beginning at 12:00 and moving clockwise, red is followed by orange, with yellow at 3:00, green at 6:00, blue at 9:00, and then indigo. Note that orange is produced by additively mixing red and yellow, while blue and red in various proportions produce indigo and violet. The color solid adds brightness and saturation to represent each of the 7 million distinct colors.

The color circle and color solid are important because they shows that we only need four primary hues of red, yellow, green, and blue, along with brightness and saturation, to produce all 7 million visible colors. That means we only need four types of cones after all, each sensitive to red, yellow, green, or blue. This is a technique employed by artists who mix colors on the palette from a relatively small set of basic pigments. And, in fact, you only need three primary colors:

• Red, Green, and Blue; or
• Red, Yellow, and Blue.
  

Television, and the computer you're using right now, relies on the RGB system to produce all the millions of colors.Technically, artists and theater lighting designers produce colors by subtractive mixture, while television and computers produce them by additive mixture.

All of this gets accounted for in the trichromatic theory of color vision, originally proposed by Thomas Young in 1802 (Young also proposed the wave theory of light, and translated the Rosetta stone) and by Hermann von Helmholtz in 1866 (in his Treatise on Physiological Optics, 1856-1867). The trichromatic theory, also known as the Young-Helmholtz theory, gets its name from the idea that there are three types of cones, each maximally sensitive to one of three primary colors:

• "red" (sensitive to long wavelengths),
• "blue" (sensitive to short wavelengths), and
• "green" (sensitive to medium wavelengths).

Any light stimulus activates each of these receptors to different degrees, depending on the mix of wavelengths in it. Pure blue light would stimulate the blue short-wave cone, but not the long-wave cone (or not very much). Purple light would stimulate both the short-wave and the long-wave cone.

Technically, color sensations aren't based strictly on the wavelength of light that falls on the retina. Hue and saturation also make a difference, as does the surrounding illumination. These facts can combine to create a number of illusions of color. But we're going to ignore them for purposes of exposition, and just focus on the relation between the wavelength of the physical stimulus and the corresponding sensation of color.

But so far, all this is logic. We've shown on physical and logical grounds that you could get color vision from just four color receptors, but we haven't shown that this is how the visual system actually does it. But would we have gone through all this if it didn't? Naaaahhhh. Still, it's important to look at the psychological evidence.

First, consider that color blindness comes in two principal forms:

Total loss of color, known as achromatopsia, is relatively rare.

This suggests that red and green are coded by the same receptor system, which in turn is different from that which codes yellow and blue. If red and green, and yellow and blue, were on different receptor systems, then we could get other combinations of color blindness.

Actually, there are two forms of red-green colorblindness, but these have almost indistinguishable effects on color vision.

• Protanopia, loss of the pigment associated with long wavelengths ("red vision");
• Deuteranopia, loss of the pigment associated with medium wavelengths ("green vision").

Blue-yellow colorblindness is known as tritanopia, and reflects loss of the pigment associated with short wavelengths ("blue vision").

The clincher comes from the phenomenon of negative afterimages. If you stare at a color patch (the more saturated the better) for a while, and then shift your gaze to a white or light-gray surface, you will see its complementary color (if you shift your eyes around the room, you will take your afterimage with you until it dissipates: it's in your eyes, not on the surface). Thus, staring at red produces a negative afterimage of green, and vice-versa; staring at yellow produces a negative afterimage of blue, and vice-versa.

The American pop and "neo-Dadaist" artist Jasper Johns uses negative afterimages to great effect in his paintings of targets and American flags.

"Target" (1974)	"Target"	"Moratorium" (1969)	"Flags" (1967-1968)

Some of these explicitly refer to negative afterimages. Note, in "Target" and "Flags" the presence of fixation points in the left and right, or top and bottom, portions of the painting. Johns once characterized his oeuvre as an exploration of "how we see and why we see the way we do" (Jasper Johns: Seeing with the Mind's Eye, exhibition at the San Francisco Museum of Modern Art, 2012-2013). Like Seurat, Johns was greatly influence by the psychology of visual sensation and perception. We'll see more of his work later.

Actually, and importantly, you can get achromatic afterimages as well: staring at a drawing of black ink on white paper will (if you do it right) produce a white-on-black afterimage.

But where does yellow come from? And the afterimages? Theoretically, yellow could come from a mixture of red and green, and this is what Young and Helmholtz thought, but the problem is that yellow is perceived as a pure color, not a mixture (remember the psychological primaries?). And the trichromatic system, itself, can't produce negative afterimages.

To cope with these problems, Leo Hurvich and Dorothea Jameson, at the University of Pennsylvania, have proposed the opponent-process theory of color vision (this is actually a more sophisticated version of a theory originally proposed by the 19th-century German physiologist Ewald Hering). The opponent-process theory holds that neural impulses arising from the rods and cones excite six neural processes (localized in the retina, optic nerve, and lateral geniculate nucleus) which are themselves arranged into three opposing pairs:

Each member of the pair serves as an antagonist of the other, so that excitation of one member inhibits the other. Thus, pure red light stimulates the long-wave cone, which sends neural impulses to the red/green neural system, exciting the red element but suppressing activity in the other. When the red light is turned off, excitation of the red element ceases, and so does the inhibition of the opposing green element; the disinhibition of the suppressed green element produces a green negative afterimage. The same thing happens with stimulation by blue light, which when it ceases produces a yellow afterimage; and stimulation by white light, which when it cases produces a black afterimage.

The antagonistic relationship between red and green, and between blue and yellow, means that, ordinarily, we cannot see red and green, or blue and yellow, simultaneously.  However, under special circumstances people can see "forbidden" colors such as greenish red, or yellowish blue, etc.  This phenomenon, while difficult to observe and replicate, has led to some revisions in the opponent-process theory of color vision -- revisions that are too technical for an introductory course.  If you're interested, see "Seeing Forbidden Colors" by Vincent A. Billock and Brian H. Tsou, Scientific American, 02/2010).

The opponent-process theory explains why colorblindness comes in the various forms that it does. The red and green processes are tied together, so that a weakness in one creates a weakness in the other as well.

The experience of color is determined by the brain, not by the stimulus or the receptor.

Thus, the 7 million colors which we can see are produced by the stimulation of rods and three kinds of cones in the retina, which in turn activate and inhibit three color systems located elsewhere in the neural system supporting vision. But where? The best evidence to date, based on research by Russell and Karen DeValois, another husband-and-wife pair of vision scientists, this time working at UC Berkeley, is that the opponent-processes themselves are located in the lateral geniculate nucleus. 

Other Specific Fiber Energies

In gustation, there appear to be five basic tastes:

All other tastes appear to be produced by mixtures of these elements.

Link to an interview with Linda Bartoshuk.

For a long time, it was thought that receptors specific to the four "basic tastes" -- sweet, sour, salty, and bitter -- were located in discrete areas of the tongue. The sensation of sour was thought to be most acute at the sides, least at the tip and base. The sensations of sweet and salty are most acute at the tip, least at the sides, and base. The sensation of bitter is most acute at the base, and on the throat and the palate. This suggests that there may be four different receptors, each sensitive to a different basic taste, and each uniquely distributed on the tongue -- after the manner of the "tongue map" proposed by E.G. Boring in 1942, based on Hanig's research.  Unfortunately, we now know that the tongue map is based on a misinterpretation of Hanig's findings.  There actually are differences in sensitivity of different areas of the tongue to the different elementary tastes, but these differences are very small -- too small to justify identifying a particular taste with a particular area. 

Moreover, histological studies have failed so far to reveal differences in the taste buds at various sites on the tongue, so the precise organization of taste remains something of a mystery. While it's true, to some extent, that different areas of the tongue are differentially sensitive to the various basic tastes,the situation is more complicated than this -- not least by the discovery of umami, that fifth basic taste, which made Boring's "taste map" more than a little quaint, much like maps of the world printed before 1492. More critically, it appears that clusters of cells, each sensitive to different chemical molecules, are dispersed widely across the tongue. Much in the manner of color vision, these clusters may be maximally sensitive to particular molecules, they are also responsive to others. And much in the manner of color vision, the output from these receptors seem to be integrated at high levels of the nervous system to yield the sensation of taste.

Our best guess now is that there is one receptor for each of four of the basic tastes: sweet, sour, salty, and umami, distributed across the tongue.  For bitter, however, there appear to be dozens of receptors; and, to make things even more interesting, they're not just found in the tongue.  Cells that respond to "bitter" tastes are found in lots of different regions of the body -- in the brain, in the respiratory system, in the gastrointestinal system, even in the genitourinary system.  It appears that these bitter receptors are part of a second kind of immune system which responds specifically to poisonous substances.  Of course, by the time a poison triggers a bitter receptor in the colon or the bladder, it's already been ingested.  But that's where this second immune system kicks in, triggering various bodily responses that fight bacteria in various ways -- for example, by flushing them out of the body in urine.  (See "Bitter Taste Bodyguards" by R.J. Lee and N.A. Cohen, Scientific American, 02/2016).

In olfaction, there appear to be six distinct odor qualities:

• spicy
• fragrant
• ethereal
• resinous
• putrid
• burned.

With respect to specific fibers corresponding to these qualities, situation seems to be rather more complicated than for taste. People can recognize about 10,000 different odors, but there are only -- only! -- about 1,000 different cells buried in the olfactory epithelium, each presumably responsive to a specific odorant molecule. In 1991, Richard Axel and Linda Buck reported that each of these cells is, in fact, sensitive to more than one chemical, though to different degrees -- in somewhat the same way that the rods and cones in the retina respond to many different wavelengths of light. The aggregate responses of these cells, then, constitute a complex -- think of a musical chord -- that represents each different identifiable odor. For their work, Axel and Buck received the 2004 Nobel Prize in Physiology or Medicine.

While people are extraodinarily good at distinguishing one odor from another, they are extraordinarily bad at identifying and naming those odors -- although this ability can improve with practice.  This fact led Yeshurun and Sobel (2010) to suggest that the primary quality of olfaction is not spicy or fragrant, etc., but rather simply pleasantness-unpleasantness.  Remember factor analysis, from the lectures on Methods and Statistics in Psychology?  When subjects were asked to describe odors, the primary factor turns out to be a dimension of pleasantness-unpleasantness.  And the same primary factor emerged when investigators analyzed the molecular features of a large number of different odorants. The implication is that pleasantness is central both to the chemical structure of the odor stimulus and the resulting sensory experience.  On the other hand, it may be that pleasantness is a kind of "superfactor" that subsumes other basic qualities, like spicy, fragrant, and ethereal vs. resinous, putrid, and burned as subordinate qualities.  Something similar has been proposed for the structure of intelligence, as discussed later in the lectures on Thinking. 

In the tactile sense, there is some controversy about basic qualities, but the following have been offered as possibilities:

In temperature, there are actually two different temperature senses:

Each is associated with a different receptor organ: cold, the Krause end-bulbs; warm, the Ruffini end-organs. The sensation of hot comes from the simultaneous stimulation of both structures.

In pain, we can distinguish between fast and slow pain, corresponding to "C" and "A-delta" fibers. We can also distinguish between sensory pain, which registers in the somatosensory cortex of the parietal lobe, and suffering, which registers in the anterior cingulate gyrus of the limbic system.

Sensory Psychophysics and Signal Detection

Work on the sensory qualities illustrates the basic psychophysical principle:

every psychological property of a sensation is related to some physical property of the corresponding stimulus.

For example, in audition:

• pitch (high or low) is related to the frequency of vibration;
• loudness (loud or soft) to the amplitude of vibration; and
• timbre (e.g., the difference in sound between a clarinet and an oboe) to the shape of the sound wave, and the number of harmonics it produces.

Each of these properties is transduced by the sensory receptors, transmitted to the cortex, and detected by the observer.

But not all stimuli are detected by the observer. Some stimulus energies don't fall on the sensory surfaces. Others do, but are too weak to be sensed by the organism. Classical psychophysics holds that detection of a stimulus is largely a function of the intensity of that stimulus. Thus, we next turn our attention to intensity -- the one quality that all sensory modalities have in common.

Thresholds

Just as each neuron has a threshold for firing, so each modality has a threshold for detection -- for conscious awareness of a stimulus event.

• The absolute threshold is the weakest stimulus that can be detected in a modality. The human sensory apparatus is remarkably sensitive: under optimal circumstances, we can see a candle flame 30 miles away on a dark night; and smell a single drop of perfume in a 6-room house.
• The relative threshold is the smallest change in intensity that can be detected -- the "just-noticeable difference" (JND) in stimulation. Notice that the absolute threshold is a special case of the relative threshold, as it represents the JND between nothing and something.

In psychophysical terms, intensity is related to the amount of stimulus energy falling on the sensory surfaces. The more intense the stimulus, the stronger the sensation.

[Reminder: intensity is coded by the summation of activity in sensory neurons: temporal, the frequency of single unit discharges; and spatial, the number of units firing simultaneously.]

The Psychophysical Laws

This would seem fairly obvious. But, in fact, there is no isomorphism between physical and sensory intensity. For example, the relative threshold -- the amount of change in physical energy needed before that change is detectable -- depends on the intensity of the original stimulus. If the original intensity is low, even a very small change is noticeable. If it is high, a relatively large change is required.

This principle is represented in Weber's Law:

dI/I = C,
where I = physical intensity and C is a constant.

In other words, the amount of intensity which must be added to a stimulus to produce a JND is a constant fraction of that intensity. To take an example, assume that Weber's Fraction, C, = 1/10. Thus,

You may wish to do some other problems like this, using different Weber Fractions and different values of I, just to get a feel for the enterprise.

Weber's Fraction differs for each sensory modality, and for each species, and provides a universal index of the sensitivity of a modality. Thus, if a modality is associated with a small Weber's Fraction, it is highly sensitive (small JNDs). In humans, vision and audition are the most sensitive modalities, taste and smell the least. But in cats, Weber's Fraction for smell is 1/300 that of humans.

A more general principle relating sensory intensity to physical intensity is Fechner's Law:

S = k(log I),
where S = sensory intensity; I = physical intensity; k = a constant; and log = a logarithmic function.

Thus, for every value of I, a corresponding value of S is given by the formula. Ignore the constant k, and assume logarithms to the base 10 (you may wish to get out your calculator and work through some other examples). Thus,

Notice that according to Fechner's Law, sensation changes more slowly than stimulation.

As stimulation grows from 1 to 200 units, sensation grows only from 0 to 2.3 units). This follows from Weber's Law: 10 additional units of stimulation makes a big difference to sensation at the bottom end of the scale, but a progressively smaller difference as we move towards the top.

If you plot these values on a logarithmic scale, you get a perfectly straight line. The early psychophysicists loved straight lines like this, because they thought they represented truly elegant mathematical laws -- showing that psychology was a real science after all.

Fechner's Law holds across a wide range of situations, but there are some exceptions. For example:

An even more general psychophysical law, Stevens' Law, handles the exceptions. In fact, Stevens' Law is so general that handles all known psychophysical relations. Stevens' Law states that:

S=k(I^N),
where S = sensory intensity; I = physical intensity; k = a constant; and N = an exponent (either power or root).

In words, Stevens' Law states that for any quality of sensation, there is an exponent that relates changes in sensation to changes in stimulation.

Get out your calculators again, and we'll work through some examples.

In fact, Fechner's Law is a special case of Stevens's Law, where the exponent, N, is less than 1.0 (an exponent of β means the square root; an exponent of 1/3 means the cube root, etc.). As an example, consider what happens when N = 1/2 (or, alternatively, 0.5):

Again, although the precise values differ from those given in the illustration of Fechner's Law, notice that big changes in I yield small changes in S, just as predicted by Fechner.

Where N = 1, you get the "length" exception to Fechner's Law, where every change in I produces an isomorphic change in S.

Where N is greater than 1.0, you get the "pain" exception. For example, assume that the exponent N = 3/2 (or 1.5):

Notice that physical intensity grew from 1 to 200 units, but sensory intensity grew from 1 to 2828 units. Thus, even a small change in I produces a big change in S.

For a long time, Stevens' Law was held to be a general psychophysical law that applied to all modalities, and all qualities within a modality, under any circumstances. It was assumed to describe the operating characteristic (an term appropriated from engineering) of the sensory transducers:

That this is not exactly the case is witnessed by the fact that there are "psychophysical" relations that follow Stevens's Law, but have nothing to do with physical sensation, or transduction. For example,

• Stevens found an exponent that related the magnitude of the crime (number of dollars stolen or people murdered) to the length of the sentence imposed on convicted criminals;
• Stevens also found another exponent that related the size of the audience to the number of speech dysfluencies (stutterings, etc.) displayed by a public speaker.
• Lewis Fry Richardson (1947) found that a power function, with an exponent less than 1), relates the severity of war (in terms of casualties) to their frequency: there are many more low-casualty wars than there are high-casualty ones.
• And in an application of "Richardson's power law", Neil Johnson and his colleagues (2011) found the same pattern in the incidence of terrorist attacks, and of insurgent attacks in the Iraq and Afghanistan wars followed the same pattern: small incidents are very frequent, large incidents (like the 9/11 attacks) are very rare.

These kinds of relations go far beyond the transduction of stimulus energies. Still, Fechner's Law (Stevens's Law with the exponent < 1) is a general principle that applies to judgments of all kinds.  You'll encounter it everywhere, if you just look for it -- frequently enough that it's easy to think of it as some kind of law of nature. 

Signal Detection Theory

More important, there evolved a gradual appreciation that the psychophysical laws themselves were misleading.

One troublesome aspect of the psychophysical laws was that they were derived from sterile laboratory conditions, in which there were no special rewards for detecting a stimulus, and no special costs for missing one, or "detecting" one that is not actually there. This is not characteristic of sensory detection in the real world, where -- at times, at least -- detection accuracy may literally make the difference between life and death.

So analyses of detection need to take both motivation and expectancy into account.

These considerations gave rise to signal detection theory, a form of psychophysics which takes into account both

In the basic signal detection experiment, a stimulus, or signal, is presented against a background of noise. Some trials present both signal and noise, others present just noise (these are known as catch trials); and the task of the subject is to say, on each trial, whether the signal is present. The task is made fairly difficult by the fact that the signal is just slightly more intense than the noise. This situation injects considerable uncertainty into each trial.

Thus, the general signal-detection experiment is structured as follows:

By comparing the rate of hits with the rate of false alarms, signal-detection theory generates two statistics:

Some signal-detection experiments employ other measures of sensitivity and criterion, but d' and β are the ones you'll encounter in most elementary discussions.

The beauty of signal detection theory is that it permits investigators to measure the observer's sensitivity independent of (i.e., uncontaminated by) his or her criterion.

Here is specimen signal-detection performance an observer with high sensitivity and no bias: his hit rate is 100% (50% hits is 100% of trials when the signal is on), and his false alarm rate is 0. For technical reasons, it is not possible to calculate d' and β under these circumstances, but both values would be very high.

Here is performance by an observer with less sensitivity to the stimulus, but still no bias: the false alarm rate is still 0, but the hit rate is now only 80% (40% hits is 80% of the trials when the signal is present). In this case,d' = .84, and β = .70.

Here is performance by an observer who has no sensitivity at all (perhaps he's blindfolded), but a strong "liberal" bias toward saying "yes", the stimulus is present, even when it's not -- generating an 80% hit rate, but also an 80% false-alarm rate. In this case,d' = 0. For technical reasons, it's not possible to calculate β, but I use the example anyway to give you an idea of what a low criterion would do to performance.

And here is performance by another observer who has no sensitivity at all, but a strong "conservative" bias toward saying "no", the stimulus is not present, even when it is -- generating a hit rate of only 30%, but, more critically, an equivalent false-alarm rate of 30%. In this case, again,d' = 0. For technical reasons, it's not possible to calculate β, but I use the example anyway to give you an idea of what a high criterion would do to performance.

Here's a more realistic example of an observer who actually has some sensitivity (the blindfold is removed, or maybe he's seeing through only one or two layers of cloth), but also a liberal response bias -- he says "yes" on 60% of the trials, but his hit rate is 80%, much greater than his false alarm rate of 40%. In this case,d' = 1.09, and β = .72.

And here's another more realistic example of an observer with some degree of sensitivity, but also a conservative response bias: he says "yes" only 30% of the time, but his hit rate of 50% is still greater than his false-alarm rate of 10%. In this case,d' = 1.28, and β = 2.27.

Within this framework, the observer's expectations are manipulated by varying the proportion of catch trials, in which the signal is actually off. If there are relatively few such trials, subjects will be biased toward saying "yes" even when they are uncertain -- after all, the signal is usually present. If there are relatively many such trials, they will be biased toward saying "no".

When the stimulus is present on 70% of the trials, the observer will tend to adopt a liberal response bias, lowering his criterion for saying "yes". Still, his hit rate of 74% is greater than his false-alarm rate of 60%.In this case,d' = 0.39, and β = .84.

But when the stimulus is present on only 30% of the trials, the observer will likely adopt a conservative response bias, raising his criterion for saying "yes". Even so, his hit rate (60%) is greater than his false-alarm rate 17%).In this case,d' = 1.21, and β = 1.53.

In addition, the observer's motivations may be manipulated by means of a payoff matrix. If there is a high payoff for hits, and no penalty for false alarms, the subject will be very liberal in saying "yes": he or she will make lots of false alarms, but that's all right. If there is a high penalty for false alarms, the same subject will become very conservative in saying "yes", because each error costs dearly.

Here is a balanced payoff matrix, in which the observer receives 25 cents for every hit, and loses 25 cents for every false alarm.

Here is a payoff matrix intended to induce a liberal response bias: the observer gets 25 cents for every hit, but loses only 10 cents for every false-alarm. Under these circumstances, assuming a balanced design with 50% catch trials, the observer will win more than he loses.

And here is a payoff matrix intended to induce a conservative response bias: the observer gets only a dime for every hit, and loses a quarter for every false alarm. Under these circumstances, the observer will be very careful, adopting a relatively high criterion for saying "yes".

Signal-detection theory has a number of important practical applications. For example, imagine a radiologist examining an X-ray, or CT or MRI scan, for evidence of a tumor (here's a mammogram). The tumor, if it's present, is a "signal" that is visible against a background of "noise" generated by other tissues in the area. So, this is a classic problem in signal detection. If the patient has a family history of breast cancer, that increases the risk that she too has a tumor, so the radiologist may adopt a relatively low criterion for detecting the tumor -- meaning, of course, that there is an increased risk of a false alarm (it might just be a benign fibroid mass). But then again, he also has to consider the costs of a false alarm (e.g., an unnecessary biopsy or exploratory surgery) against the costs of a miss (e.g., that a tumor goes undetected, and thus untreated, and metastasizes to other parts of the body). So, again, the performance of a radiologist on a task like detecting tumors is a matter, not just of the physician's sensitivity, but also of the criterion he or she has set, based on expectations and goals.

SDT also has a number of theoretical implications. In the first place, it denies that there is such a thing as absolute sensitivity. Sensitivity varies with expectations and motives. This undercuts the older view of the passive observer, as discussed in the lectures on learning: the stimulus is transduced by the receptor organ, transmitted through the nervous system, and -- provided it is intense enough -- detected by the observer. And it substitutes a view of the active observer: the stimulus is transduced and transmitted as before, but the observer must make a judgment about the stimulus: whether it is present, or whether it has changed. These decisions are influenced by the person's goals and expectations.

"Subliminal" Perception

If, indeed, there is no such thing as absolute sensitivity, maybe the whole idea of sensory thresholds is misleading as well. The concept of the sensory threshold comes to us from Johann Herbart (1811), who proposed that every sensation must cross a limen (German for "border", or "threshold". derived from the Latin limes, meaning frontier) in order to be represented in consciousness. But Herbart suggested that sensations that do not cross the threshold are still present in the nervous system -- even though they don't influence behavior.

The very first experiment on subliminal perception was, in fact, the very first psychological experiment published in the United States -- in 1884, by Charles S. Peirce (pronounced "Perss"), a philosopher at Johns Hopkins, and Joseph Jastrow, his graduate student (and the first person to receive a PhD in psychology from an American university). Peirce and Jastrow were interested in Herbart's concept of the limen, and they wanted to determine whether there was any such thing -- that is, a level of stimulation so weak that it has no effect on the perceiver.

In their experiments, an observer was required to distinguish between the heaviness of two weights, or the brightness of two lights -- thus, their experiments were actually concerned with difference thresholds. In addition to making the judgment of relative weight (or brightness), the observers were also asked to rate their confidence in their judgment. As the difference between the two weights diminished, so did the observers' confidence in their judgments. But even when the observers had zero confidence in their judgments, meaning that they had no conscious appreciation of the difference between the two stimuli, still their forced-choice judgments were more correct than would be expected by chance.

Here's an instance of the value of statistical analysis. We could debate all day whether a 60% hit rate was larger than the 50% hit rate expected by chance. But Peirce and Jastrow actually devised new statistical techniques to demonstrate that this difference was, indeed, statistically significant.

Later, the concept of subliminal perception became quite controversial, with some commentators (like Vance Packard) warning about the threat of "hidden persuaders", and others asserting that subliminal perception didn't really occur, and that investigators like Peirce and Jastrow just didn't establish their thresholds rigorously enough. Over the past 100 years, however, a large number of experiments like Peirce and Jastrow's have been performed, using a variety of different methodologies.

In all of these paradigms, the stimulus has some effect on the perceiver's experience, thought, or action, even though the perceiver does not consciously perceive the stimulus. This is not a matter of simple guessing, because in these cases the hit rate is greater than the false-alarm rate -- that is, in the terms of signal-detection theory,d' is above zero.

Because in many of these cases the stimulus in question is not truly subliminal, in the sense of being weak, but rather is brief, or masked, or simply unattended, the concept of subliminal perception has been replaced by the concept of implicit perception (Kihlstrom et al, 1992).

It turns out that explicit and implicit perception can be dissociated, so that the stimulus can affect the perceiver's behavior even though it is not consciously perceived. That is what we mean, these days, when we refer to "subliminal" perception -- even though, technically, the stimulus may not be truly subliminal, it is still processed outside conscious awareness.

Implicit perception is real, and it's interesting, but that doesn't mean that it's all that powerful. Peirce and Jastrow's observers were right only about 60% of the time, after all. In fact, truly subliminal effects on perception are relatively weak, and analytically limited. There's only so much you can do with a subliminal stimulus; anything more requires conscious perception. There's no reason to fear "subliminal" advertising, or any other form of hidden persuasion. Subliminal perception can't make you go out and buy a Coke at the movies -- but it might influence you to buy a Coke, rather than a Pepsi, if you're already thirsty.

This page last revised 01/01/2024.