Like Anderson's cognitive algebra, Kenny's Social Relations Model is more a method than a theory.  The round-robin design, coupled with sophisticated statistical tools, can be used to partition person perception into its various components, and so to answer a wide variety of questions about a wide variety of topics in social cognition. 

Person Perception as Perception

Research on impression-formation, from Asch (1946) to Anderson (1974) and beyond, has largely made use of trait terms as stimulus materials.  This is certainly appropriate, because -- as Fiske & Cox (1979) demonstrated, as if we needed any proof -- we often describe ourselves and others in terms of traits.  Working with traits injects substantial economies into impression-formation research, because they're easy to manage.  Moreover, traits may fairly closely represent the way information about people is stored in social memory. But it's also clear that an exclusive focus on traits can give a distorted view of the process of impression formation, because traits are not really -- or, at least, not the only -- stimulus information for social perception.  

We don't walk around the world with our traits listed on our foreheads, to be read off by those who wish to form impressions of us.  Rather, the real stimulus information for person perception consists of our physical appearance, our overt behavior, and the situational context in which they appear.  Accordingly, in addition to describing how we make use of trait information to form impressions of personality, a satisfactory account of person perception needs to answer a different sort of question -- to wit:

How do we get from the physical stimulus of the person -- his or her appearance and behavior -- to his or her mental state?

Or, put another way,

What features of the physical stimulus give rise to our impressions of a person's mental state?

Again, the same question about perception occurs in the social domain as in the nonsocial domain.

• In the nonsocial domain, the problem of perception is to unpack physical stimulus information to make perceptual inferences about the object's physical state -- it's form, location, activity, and affordances of the distal stimulus.
• In the social domain, the problem of perception is to unpack physical stimulus information to make perceptual inferences about a person's mental state -- his or her thoughts, feelings, and desires.

In the nonsocial domain, the stimulus information for perception consists of patterns of physical energy (the proximal stimulus) radiating from the distal stimulus, and impinging on the perceiver's sensory surfaces.  In the social domain, the stimulus information for perception consists of a person's surface appearance and overt behavior.  These include, among others, the person's:

• facial expressions;
• bodily orientation, posture, and movement;
• vocal cues;
• interpersonal distance;
• eye contact and touching;
• physical appearance, dress, and cleanliness; and
  
• the person's local behavioral environment in which we encounter the person (particularly those aspects of the situation that are under the person's control).

Physiognomy The use of physical features to make inferences about character and personality has its roots in physiognomy, a pseudoscience in which a person's character was judged according to stable features of the face -- much as the 19th-century phrenologists judged character from the bumps and depressions on the skull.   The word physiognomy comes from the Greek Physis (nature) and gnomon (judge), and began with the observation that some people looked like certain animals.  It was only a short step, then, to infer that those individuals shared the personality traits presumed to be characteristic of those animals.  More generally, physiognomy was based on the assumption that a person's external appearance revealed something about his internal personality characteristics.   References to physiognomy go back at least as far as Aristotle, who wrote in his Prior Analytics (2:27) that It is possible to infer character from features, if it is granted that the body and the soul are changed together by the natural affections... passions and desires.... Aristotle (or perhaps one of his students) actually produced a treatise on the subject, the Physiognomonica. Physiognomy fell into disrepute in the medieval period, and was revived by Giambattista della Porta (De humana physiognomia, 1586), Thomas Browne (Religio Medici, 1643), and Johann Kaspar Lavater (Physiognomische Fragmente zur Beforderung der Menschenkenntnis and Menschenliebe, 1775-1778).  And it's been revived again, much more recently.  In a study of transactions on a peer-to-peer lending site (Prosper.com), Durate (2009) showed that people could make valid judgments of trustworthiness based on a head-shot: the criterion was the applicant's actual credit rating and history.
Here are some physiognomic drawings by Charles LeBrun (1619-1690) a French artist who helped establish the "academic" style of painting popular in the 17th-19th centuries (from Charles LeBrun -- First Painter to King Louis XIV).

Facial Expressions of Emotion

A good example of social-perception research involving descriptions of physical stimuli is the work of Ekman (1975, 2003; Ekman & Friesen, 1975) and others on facial expressions of emotion.  In his work, Ekman has been particularly concerned with determining the "sign vehicles" by which people communicate information about their emotional states to other people.  The fact that such communication occurs necessarily entails that there is a receiver who is able to pick up on the communications of a sender -- and this information pickup is exactly what we mean by perception. 

One of Ekman's most famous findings is that people can reliably "read" certain emotions from the expressions on people's faces.  This is true even when the sender and receiver come from widely disparate cultures.  Close analysis of these expressions shows that each of them is comprised of a particular configuration of muscle activity.  These include:

• The type (or topography) of action (brow raise, nose wrinkle, lip corners down, etc.);
• the intensity of action (the magnitude of the change in physical appearance resulting from an action); and
• the timing of action (abrupt or gradual speed of onset, short or long duration, speed of offset, etc.).

Facial Expression	Corresponding Emotion	Analysis
	Happiness
	Sadness
	Fear
	Anger
	Surprise
	Disgust

Ekman's system for coding the facial musculature is known as the Facial Action Coding System (FACS).  The system has more than 60 coding categories for various muscle action units (like the Inner Brow Raiser or the Lip Corner Puller) and other action descriptors (such as Tongue Out or Lip Wipe).  Each basic emotion, and every variant on each basic emotion, can be described as a unique combination of these coding categories.  And each coding category is associated with a specific pattern of muscle activity.

Cross-cultural studies show that Ekman's basic emotions are highly recognizable across cultures.  Nelson and Russell (2013) summarized several decades' worth of such studies, involving subjects from literate Western cultures (mostly, frankly, American college students), literate non-Western cultures (e.g., Japan, China, and South Asia), and non-literate non-Western cultures (e.g., indigenous tribal societies in Oceania, Africa, and South America).  Subjects from all three cultures recognized prototypical displays of the six basic emotions at levels significantly and substantially better than chance.  This is consistent with the hypothesis that the basic emotions, and the apparatus for producing and reading their displays on the face, is not a cultural artifact but something that is, indeed, biologically basic.

FACS is intended for use by professional researchers and clinicians, including computer analysis of facial expressions.  But the fact that people can reliably read emotions from other people's faces suggests that our perceptual systems are sensitive to changes in facial musculature. Ekman's FACS system is a formal description of the physical stimulus that gives rise to the perception of another's emotional states.

The stimulus faces used in much of Ekman's research comes from actors posing various expressions according to Ekman's instructions, but we can read emotional states from people's faces in other circumstances as well.  

Consider, for example, photographs taken in April 2000, when Elian Gonzalez, a Cuban boy who had lost his mother during an attempt to escape from Cuba, was being sheltered by some of his mother's relatives in Miami.  Elian's father, who was estranged from his mother, and had remained in Cuba, demanded that he be returned to Cuba.  The US Department of Justice, for its part, determined that, legally, custody of the boy should be given to his closest living relative -- his father, who was in Cuba.  The Miami relatives refused to turn Elian over for repatriation, and in the final analysis an armed SWAT team from the Border Patrol forced its way into the relatives' house to retrieve the boy.  Little did the officers know that a newspaper reporter and photographer were already in side the house.  The resulting remarkable sequence of photographs shows the surprise of one officer when he discovered the photographer in the room.

The Embodied Smile

Like Ekman, Paula Niedenthal and her colleagues have cataloged a number of different types of smiles, but the differences they observe go far beyond patterns of facial activity..  It turns out (to quote the headline in the New York Times over an article by Carl Zimmer,  01/25/2011), that there's "More To a Smile Than Lips and Teeth".  Some smiles are expressions of pleasure, while others are displayed strategically, in order to initiate, maintain, or strengthen a social bond; some smiles comprise a greeting, others display embarrassment -- or serve as expressions of power.  Niedenthal has been especially active in examining the process of smile recognition -- recognizing the differences among smiles of pleasure, embarrassment, bonding, or power.  She proposes that smiles are "embodied" in perceivers through a process of mimicry, in which different types of smiles initiate different patterns of brain activity in the perceiver -- patterns that are similar to those in the brain of the person doing the smiling.

In one experiment, Niedenthal found, not surprisingly, that subjects could accurately distinguish between these different types of smiles.  But when they held a pencil between their lips, essentially interfering with the facial musculature that would mimic the target's smile, accuracy fell off sharply.  Similarly, the judgments of subjects in the "pencil" condition were more influenced by the contextual background, than by the smiles themselves.  Apparently, mimicry plays an important role in the recognition of smiles (and, probably, other facial expressions as well).

Niedenthal's work exemplifies a larger movement in cognitive psychology known as embodied cognition or grounded cognition.  For most of its history, psychology has assumed that the brain is the sole physical basis of mental life. Embodied cognition assumes that other bodily processes -- in this case, the facial musculature -- are also important determinants of mental states.  And so is the environment -- like the context in which a smiling face appears.  Proponents of embodied cognition do not deny the critical role of the brain for the mind.  They just argue that other factors, in the body outside the brain, and in the world outside the body, are also important.

Full disclosure: Prof. Niedenthal worked in my laboratory as an undergraduate.  But even so, her work on the smile is the most thorough analysis yet.  For an overview of her work on smiles, see P.M. Niedenthal et al. "The Simulations of Smiles (SIMS) Model: Embodied simulation and the Meaning of Facial Expression", Behavioral & Brain Sciences, 33(6), 2010.

Emotion Perception Beyond the Face

The face is a major channel for communicating emotional states, and probably the most important, but it is not the only one.  Tone of voice, gesture, posture, and gait are also available channels -- although they have not been given as much systematic attention as the face.

The importance of nonfacial expressions of emotion is underscored by cases of Moebius Syndrome, a congenital condition first described by Paul Julius Moebius in 1888.  The condition entails a paralysis of the facial musculature (the illustration shows Kathleen Bogart, who has Moebius syndrome, and who studies the disorder, with her husband, Beau, from the New York Times, 04/06/2010).  People with Moebius syndrome cannot express emotions on their faces, so they must find other means of emotional expression, including both verbal and nonverbal channels.  They have no difficulty recognizing other people's facial expressions, however.  And they still feel various emotions.  This is important, because a major theory of emotion communication implies that we mimic other people's facial expressions, and feedback from our own facial expressions shapes both our perception of their emotional states, and our own emotional experience.  That can't happen in cases of Moebius syndrome, of course, because the facial paralysis prevents the feedback.  so either mimicry isn't important, or there are other mechanisms for emotion perception. 

A similar problem is encountered in Bell's palsy, a neurological condition involving the usually) temporary paralysis of the facial musculature caused by inflammation of the VII cranial nerve.  Jonathan Kalb, a theater professor at Fordham University, has written about his own experience with Bell's palsy in "Give Me a Smile" (New Yorker, 01/12/2015).  He has never completely recovered from the illness, with the result that his smile is "an incoherent tug-of-war between a grin on one side and a frown on the other: an expression of joy spliced to an expression of horror).  Kalb reports that he has difficulty communicating positive affect to other people, and they have difficulty reading positive affect from his facial expressions.  He also suggests that, because of disrupted feedback from the facial musculature, he has diminished experience of pleasant affect, and must engage other, compensatory strategies -- some drawn from tricks employed by Method actors.

The Determinants of Physical Attractiveness

Another prominent topic for person perception research has to do with the perception of facial beauty.  We know from research on interpersonal attraction that physical attractiveness is the most powerful determinant of likeability (e.g., Berscheid & Walster, 1974).  And we also know that likeability -- evaluation, in Anderson's terms -- influences a host of social judgments through the halo effect.  But exactly what determines physical attractiveness remains a mystery.  As Berscheid and Walster (1974), two social psychologists who are probably the world's foremost experts on interpersonal attraction, concluded, "There is no answer to the question of what constitutes beauty". 

Why is this question important?  One reason is Thorndike's halo effect.  People tend to believe (regardless of whether it's actually true) that socially desirable features go together.  Therefore, if someone is physically attractive, they'll also tend to think that they're socially attractive -- on the "warm" end of the social good-bad scale, and on the "intelligent" end of the intellectual good-bad scale.  As the English Romantic poet John Keats wrote (in Ode on a Grecian Urn, 1819), "Beauty is truth, truth beauty".

Interestingly, there may also be a reverse halo effect.  Vincent Yzerbt, Kocolas Kervyn, and their colleagues have found that, when comparing people with each other, a person (or group) who receives high ratings on warmth may receive low ratings on competence, and vice-versa.  Apparently, the traditional halo effect occurs when evaluating individuals separately, while the reverse halo effect occurs when comparing one individual with another. 

There's no question about the bias toward facial attractiveness -- and not just in bars and bedrooms.  A number of writers have commented on the pervasiveness of "lookism", a concept modeled on racism, having to do with discrimination against those who are less than a perfect "10" (to use the title of a 1979 movie on this theme starring Dudley Moore, Julie Andrews, and Bo Derek as the eponymous beauty.  For more on lookism, see the following books, discussed by Rachel Shteir in "Taking Beauty's Measure" (Chronicle of Higher Education, 12/16/2011):

• Hope in a Jar: The Making of America's Beauty Culture by Kathy Peiss (1998).
• The Beauty Bias: The Injustice of Appearance in Life and Law by Deborah L. Rhode (2010).
• Beauty Pays: Why Attractive People Are More Successful by Daniel S. Hamermesh (2011). 
• Erotic Capital: The Power of Attraction in the Boardroom and the Bedroom by Catherine Hakim (2011).
• Pricing Beauty: the Making of a Fashion Model by Ashley Mears (2011).
• And last, but by no means least, Lip Service: Smiles in Life, Death, Trust, Lies, Work, Memory, Sex, and Politics by Marianne LaFrance (2011), a social psychologist and director of the Women's Studies Program at Yale.

The Role of Averageness

Actually, maybe there is.  A large body of literature now strongly suggests that attractiveness is strongly related to averageness -- in other words, that we find most attractive those faces (and, for that matter, bodies) that are close to the average for the population.  As counterintuitive as that may seem, there are actually good reasons to think that average faces really are highly attractive.

• According to evolutionary theory, natural selection has a normalizing function.  Because (again according to evolutionary theory) extreme values on any feature mark genetic mutations, features that are at or near the average mark reproductive fitness.  Therefore, mate selection (which, according to evolutionary theory, is all about reproductive fitness) will prefer those with average features.
• According to prototype theory, category prototypes may be thought of as the average of the instances of a category.  We respond to category prototypes as if they were familiar -- which they are, because they look like so many category instances; and we tend to prefer the familiar to the unusual.

Theory aside, a study by Langlois and Roggman (1990) does indicate that, as an empirical fact, average faces are more attractive.  In this study, full-front face-and-neck photographs of people bearing a pleasant, neutral expression (with background and lighting controlled) were digitized.  Each face was matched on the location of the eye pupils and the lip midline, and then composites were created through a computer averaging program.  The results of the study were clear: composite faces were preferred to individual faces, and the more faces that went into the composite (from 2 to 32), the more the composite face was preferred.  The more a face reflects the average of all faces, the more attractive it is.

  

Referring to the Berscheid and Walster (1974) quote above, Langlois and Roggman (1990) concluded that the question of facial beauty had been solved: [A]ttractive faces... represent the central tendency or the averaged members of the category of faces".

Averageness or Symmetry? 

But the question isn't entirely resolved, because evolutionary psychology has a somewhat different answer to the question of why we prefer average faces.  According to evolutionary psychology, patterns of experience, thought, and action that were adaptive in our ancestral environment (the Environment of Early Adaptation -- roughly the East African savanna during the Pleistocene era) have been preserved in current members of the human species through natural selection.  In this view, mate selection prefers healthy, fecund mates: facial symmetry is a marker of health and fecundity, while fluctuating asymmetries on the face (and elsewhere on the body are signs that the organism is unhealthy, and less desirable from the point of view of reproductive fitness.  Averaging eliminates these fluctuating asymmetries, and produces symmetrical faces.  So, according to evolutionary psychology, average faces are not attractive because prototypes seem familiar, but because average are more symmetrical.

There's certainly anecdotal evidence in favor of a connection between symmetry and attractiveness.  Queen Nefertiti, wife and co-ruler of ancient Egypt with the Pharaoh Akhenaten (14th century BCE) was widely acclaimed as the most beautiful woman in the ancient world (this was before Helen of Troy): her name Nefertiti even means (in rough translation) "The perfectly beautiful woman has come".  And Nefertiti is portrayed in images that survive from her time as having a perfectly symmetrical face.  Of course, these are only images -- we don't know what she really looked like.  She might just have had a good public-relations firm.

But we do know what the actress Elizabeth Taylor (who died in 2011 at age 79) looked like, and she really did have a perfectly symmetrical face -- and was universally acknowledged as fabulously beautiful.

On the other hand, 20th-century culture gives us lots of examples of very attractive woman who have prominent fluctuating asymmetries on the face: consider, for example, the prominent "beauty marks" on the faces of Marilyn Monroe and Cindy Crawford.  Beauty marks are called "beauty marks" precisely because they enhance the person's facial beauty, but as fluctuating asymmetries they're supposed to mark a lack of reproductive fitness, and thus make the person less attractive, not more.  

So something's wrong with the evolutionary argument.  In fact, the evolutionary story, like many of the "just-so" stories that abound in evolutionary psychology, sounds good, but doesn't stand up to close scrutiny.

In the first place, the connection between facial attractiveness and reproductive fitness appears to be pretty weak, perhaps nonexistent.  Kellick, Zebrowitz, Langlois, and Johnson (1998) analyzed data from the Intergenerational Studies conducted by the Institute for Human Development at the University of California, Berkeley, which included data from a large group of individuals who were born in the Berkeley-Oakland area between 1920 and 1929.  These subjects had been photographed as adolescents, and health assessments had been made on them during adolescence (ages 11-18), middle age (30-36), and old age (56-66).  There was essentially zero correlation between facial attractiveness, rated from the adolescent photographs, and health at any stage of life.  So, attractiveness does not seem to serve as a marker of health -- and thus of reproductive fitness.  Men aren't attracted to women because they think they'll produce lots of healthy babies.  Men are attracted to women because -- well, they're attractive.

In the second place, the relation between averageness and attractiveness does not appear to be mediated by symmetry.  In another experiment, Rhodes, Sumich, and Byatt (1999) employed computer-averaged composites of facial photographs that varied in their averageness, as defined in the Langois et al. (1990) study.  Subjects then rated these photographs on symmetry, pleasantness, and attractiveness.

• The raw (zero-order) correlation between attractiveness and averageness (r = .77) was much higher than the corresponding correlation between attractiveness and symmetry (r = .43).
• Using a statistical technique called the partial correlation, Rhodes et al. recalculated the correlation between attractiveness and averageness, controlling for symmetry; and also the correlation between attractiveness and symmetry, controlling for averageness.  Both correlations dropped a little, as partial correlations do.  But the important point is that averageness continued to correlate with attractiveness, even with symmetry partialled out.  Therefore, the correlation between averageness and attractiveness was not an artifact of the symmetry of average faces.
• Similar findings were obtained for pleasantness of emotional expression.     

Rhodes et al. concluded that averageness (the opposite of distinctiveness), symmetry, and pleasantness each make an independent contribution to physical attractiveness.  We find pleasant, symmetrical, average faces attractive, but the attractiveness of average faces is not an artifact of their symmetry.  (Nor, Rhodes et al. argued, is it an artifact of blending, which tends to remove distinctive characteristics even if they are symmetrical.)

Rhodes et al. (1999) assert that their experimental results "settle the dispute" between averageness and symmetry.  But the question remains why average faces are attractive.  Their best guess (and mine) is that average faces look like lots of other faces, and so they seem familiar; and we know from the mere exposure effect that we find the familiar more attractive than the unfamiliar.

Beyond Averageness and Symmetry

As the examples of Marilyn Monroe and Cindy Crawford suggest, there's probably more to facial beauty than averageness and symmetry.  In addition to those "beauty marks", there's skin tone, body-mass index and waist-to-hip ratio -- and a genuine smile.

"Babyfacedness"

Another facial feature that has been studied by researchers of person perception is babyfacedness.  Ethologists such as Konrad Lorenz have long noted that immature organisms, whether mammals or even birds and reptiles, share certain features in common:

• enlarged eyes and lips;
• soft, chubby cheeks;
• fine eyebrows;
• pug nose;
• large cranium, relative to the face; and
• non-sloping forehead.

Lorenz suggested that babyfacedness constitutes a "universal stimulus" that elicits care-taking behaviors, and inhibits aggression.  Leslie Zebrowitz (nee McArthur) picked up on Lorenz's ideas, and has studied the consequences of babyfacedness for social perception and social interaction.

Using computer "morphing" programs, it is possible to take line drawings or photographs of faces and adjust their features to make them appear more or less baby-like.  Subjects then rate the targets for various personality traits.  Research by Zebrowitz and her colleagues generally finds that people perceive baby-faced individuals as warmer, weaker, more naive and trusting; they are also more likely to help baby-faced people, even when help isn't needed.

In one study, Friedman and Zebrowitz (1992) took schematic drawings of male and female human faces and manipulated their facial features to create or erase aspects of babyfacedness.  As it happens, there is a sex difference here, with the typical female face possessing more "baby-faced" features than the typical male face.  Therefore, by adding baby-faced features they made the typical male face more "babyish" in appearance, and the typical female face appear more "mature".  They then had male and female subjects view the sketches, and make ratings of their impressions of the targets' personalities.

Baby-faced males and females alike were rated lower on power, compared to their mature-faced counterparts.  But because the typical male face has more mature features than the typical female face, the typical male was rated as more powerful than the typical female.

 

 

Baby-faced females (but not baby-faced males) were rated higher on warmth than their mature-faced counterparts.  Again, because the typical male face has more mature features than the typical female face, the typical male was rated as less warm than the typical female.

 

 

Perhaps not surprisingly, because of the sex difference in babyfacedness, baby-faced males and females alike were rated lower on masculinity (and thus higher on femininity), compared to their mature-faced counterparts.  Again, because the typical male face has more mature features than the typical female face, the typical male was rated as more masculine than the typical female.

 

Baby-faced females (but not baby-faced males) were rated more likely to be the "child caretaker" in the family higher on warmth than their mature-faced counterparts.  Again, because the typical male face has more mature features than the typical female face, the typical male was rated as less likely to be a child caretaker than the typical female.

 

Baby-faced females (but not baby-faced males) were rated less likely to be the "financial provider" in the family than their mature-faced counterparts.  Again, because the typical male face has more mature features than the typical female face, the typical male was rated as more likely to be a financial provider than the typical female.

 

These are social stereotypes, of course, but that's the point: social perceivers use the physical properties of the face to make inferences about the emotions and dispositions of the person.

The baby-faced stereotype is so commonly held that it has been employed in cartoon characters (such as Elmer Fudd and Tweetybird).  And in political humor as well.  After Vice President Cheney was involved in a quail-hunting accident, in which he peppered one of his companions with bird shot, he was depicted as Elmer Fudd -- a clear contrast between the befuddled lovableness of the cartoon character and Cheney's own reputation as a humorless right-wing ideologue.  

Of course, "baby-facedness" is a stereotype, and stereotypes can be misleading, and sometimes downright wrong.  For example, Umar Farouk Abdulmutallab, the "Underpants Bomber" of Christmas Day, 2009, was commonly described in press accounts as "baby-faced".

Beyond the Face in Person Perception

Most research on social perception has focused on the face, which is after all the most salient, perhaps the quintessential, social stimulus.  However, other nonverbal cues play a part in person perception, including vocal (prosodic) cues, gestures, and other aspects of body language.  Here, in a famous photograph from the New York Times (1957), the future president Lyndon B. Johnson (then Majority Leader of the United States Senate) discusses a point of legislation with a colleague.

Body Language

Edward T. Hall first drew attention to several aspects of body language in his popular book, The Hidden Dimension (1966).

• Proxemics, or the study of social distance.
• Posture
• Calypsis, or the strategic covering and uncovering of body parts.
• Gesture

Robert Rosenthal and his colleagues have developed a psychological test to assess individual differences in people's sensitivity to nonverbal cues -- including, but going beyond, facial cues (Rosenthal, Hall, DiMatteo, Rogers, & Archer, 1979).  The Profile of Nonverbal Sensitivity (PONS) consists of 220 2-second audio/video clips portraying a 24-year-old woman (Judith Hall, now a Professor of Psychology at Northeastern University) acting out a set of 20 vignettes.  The subjects' task is to guess which of two vignettes is being acted out. 

The vignettes are classified into a 2x2 scheme crossing positive-negative with dominant-submissive, with 5 scenes in each category.

• Positive/Dominant: e.g., admiring the weather, or talking about a wedding;
• Positive/Submissive: e.g., asking for a favor, or helping a customer;
• Negative/Dominant: e.g., threatening someone, or nagging a child; and
• Negative/Submissive: e.g., asking forgiveness, or returning a faulty purchase.
  

• 3 Visual Channels Alone: full figure, face alone, or body alone, all with no vocal cues.
• 2 Vocal Channels Alone: there are no visual cues at all; moreover, the content of the actor's speech has been rendered unintelligible by one of two methods: electronic filtering of certain frequencies or (literally) cutting up the audiotape and splicing it back together randomly.  In either case, only the "tone of voice" remained.
• 6 Combinations of Each Visual and Each Vocal Channel.

Rosenthal and his colleagues were interested in using the test to measure individual differences in sensitivity -- i.e., in perceptual ability to various channels of nonverbal communication.  In the context of this course, the PONS is a good illustration of the point that there are physical sources of social information beyond the face, including vocal and gestural cues.

Person Perception Beyond the Body

Person perception is shaped by the person's physical features, but it is also influenced by aspects of his or her dress -- what one hides and reveals, what one draws attention to are also stimulus cues as to a person's internal psychological state.  Even a person's office or bedroom can provide clues to his or her personality.

Lie Detection as a Problem of Person Perception

Much of Ekman's work on facial emotion, and much of the interest in nonverbal communication generally, has to do with the detection of deception -- or, put bluntly, with lie detection.  How can we know when someone is deceiving us?  Note that, in terms of person perception, the question of deception is this: how can we know that a person is deceiving us about his or her internal mental state -- about what he or she is thinking, feeling, or desiring?  Ekman's work on behavioral (as opposed to physiological) lie-detection has been extremely influential. He has consulted with law-enforcement agencies at all levels of government, and has even "gone Hollywood" as a consultant to the TV show Lie to Me (Fox), about Dr. Cal Lightman (played by Tim Roth), a "human polygraph" who can read body-language "micro-expressions". 

The problem, of course, is that people rarely tell us that they are lying -- what would be the point of that?  (Even Epimenides, the Cretan philosopher of the 6th-century BC, who asserted that "All Cretans are liars", could not have been lying, because if all Cretans really were liars, he -- a Cretan himself -- would have been telling the truth, thus disproving his own statement).  Instead, we usually have to infer, from their nonverbal behavior, that their verbal communications are not accurate.

And let's be clear -- lying is a serious problem of social perception.  DePaulo et al. (1996) conducted a study of everyday lying by means of diary study in which subjects were asked to keep track of all of their social interactions for a week, including instances in which they lied.  They found that lying is a common feature of social interaction.  College students recorded lying about twice a day, on average, in 1/3 of all their social interactions.  A community sample lied somewhat less often: about once a day, in about 1/5 of their interactions.  DePaulo et al. hasten to point out that most of these lies were trivial, but they were untruths nonetheless.

Can "Only a Few" "Tell a Liar"?

It turns out that we are surprisingly bad at this.  Our poor lie-detection abilities were dramatically illustrated in a study by Ekman and O'Sullivan (1991).  For this study, they created 10 1-second video clips, showing the full head-on view of a target's face and body.  Then the target described his or her positive emotions as s/he was viewing a video.  Half of the targets were viewing a pleasant nature scene, in which case s/he was telling the truth about his/her emotional state.  The other half of the targets were actually viewing a very gruesome scene -- in which case, s/he was not telling the truth.  The subjects' task was to identify which of the targets were telling the truth, and which were lying.  Ekman and O'Sullivan tested several different groups of subjects, ranging from college students and psychiatrists to law-enforcement officials. 

Averaged across all the groups, the subjects were only about 57% correct -- barely above chance levels.  Only agents of the United States Secret Service, a branch of the Treasury Department that has responsibility for protecting the President and other high officials, were particularly good at picking out liars: 53% had 70% or greater accuracy, compared to 50% "chance" level. 

A later study by Ekman, O'Sullivan and Frank (1999), focused on subjects who had special professional interests in lie-detection yielded similar, if somewhat better, results.  This time, the subjects achieved about 63% accuracy -- which is better than chance, but not all that great.  The top scorers (those with 70% or greater accuracy) were federal "law enforcement" officers, most of whom were actually agents of the Central Intelligence Agency.

So while most people are pretty bad at behavioral lie-detection, some people are better than others.  Ekman argued that lie-detection is possible when perceivers pick up on the leakage of nonverbal cues.  For example, people tend to display "Duchenne" smiles when telling the truth, but "Pan American" smiles when telling lies.  Their vocalizations also tend to show an increase in fundamental pitch.  Ekman and his colleagues were able to detect this leakage through special means, such as viewing the videos at slow-motion and noticing micro-expressions of affect that are incongruent with the content of the target's message.  However, these micro-expressions can also be picked up in real time, especially by people -- like Secret Service and CIA agents, perhaps -- who have had a lot of experience with distinguishing truths from lies.  In the 1991 and 1999 experiments, the successful subjects were able to pick up on these instances of leakage.

However, the situation is a little more complicated than this, because even some of the "top scorers" didn't perform better than chance.  This is a little counterintuitive, because you'd think that anything better than 50% would count as "greater than chance".  But as Nickerson and Hammond (1993) pointed out, when the probability of a hit p equals probability of a miss q, even 8 hits out of 10 is not significantly greater than chance with p < .05 (actually, it just misses). 

• Using a more stringent criterion of 8/10 hits, the proportion of high-scoring Secret Service agents in the Ekman & O'Sullivan (1991) study fell from 53% to 29%.  That might be a better rate than the ordinary  person on the street, but it's nothing to write home (or a paper!) about.
  
• And in the Ekman et al. (1999) study, the percentage of high-scoring Federal officers would fall from 74% as well (Ekman et al. don't provide data that would permit the actual calculation). 
  

The detection of deception can be construed as a problem for signal detection theory.  In contrast to traditional analyses of accuracy, which focus on hits (and their obverse, misses), signal-detection theory focuses on hits and false alarms -- in this case, instances where a target is called a liar but is actually telling the truth.  If you call everyone a liar, you'll correctly identify every actual liar, but you'll also misidentify all the truth-tellers.  Good lie-detection will maximize hits while minimizing false alarms. 

A bigger problem has to do with the measure of "accuracy" employed in these Ekman studies, which takes only correct responses into account.  For example, in the studies described, a subject would "catch" 100% of the liars simply by calling everyone a liar.  So it's important to take error into account.  From this perspective, we can classify subjects' responses into four categories:

Apparently, if most people are bad at detecting lies, most of us are pretty good at lying.  In fact, maybe that's why we're so bad at lie-detection: it's not so much that we're bad at detecting lies, but that we're so good at lying undetectably! 

Or, put another way, lie-detection is a problem of signal-detection, and people are typically bad lie-detectors because so often there is no signal to detect!

Lie-Detection in Lab and Life

Ekman and O'Sullivan based their conclusions about people's lie-detection abilities on their own studies -- where, frankly, the experimental procedures are somewhat informal (the subjects are typically members of the audience to whom Ekman is giving a talk).  More systematic laboratory research comes to much the same conclusion: People just aren't particularly good at it.

In their reviews of the experimental literature, Kraut (1980), Vrij (2000), and Bond and DePaulo  (2006) all found that the average receiver was barely better than chance at detecting lying under natural conditions -- that is, when the senders included both leaky and non-leaky liars.  Things looked a little better, though, when B&DeP looked at continuous ratings of honesty, rather than dichotomous judgments of lying.  Under these circumstances, honesty ratings distinguished between liars and truth-tellers to a modest degree.

Part of the problem with lie-detection may be that most of us may tend to assume that people are telling the truth.  B&DeP discovered a small "truth bias" -- to judge that people are telling the truth, even when they're not.  This effectively reduces our ability to make correct judgments of the matter.

B&DeP also examined a number of other variables that might affect lie-detection performance:

	There is a modality effect, such that lie-detection is more accurate when receivers have access to an audio channel, as well as -- or instead of -- a video channel.  This may also help explain Ekman & O'Sullivan's results: because of their emphasis on facial cues, they typically present subjects with only a visual channel.
	Detection of deception does not depend on whether the sender is highly motivated to be believed.  In fact, there is a paradox of motivation such that the receiver's "truth bias" is reduced when the sender is highly motivated to deceive.  Still, the receivers are not very accurate at correctly detecting deception.   Nor does it depend on whether the sender has been given an opportunity to prepare to deceive.  Nor does the receiver's expertise matter much (remember, even Ekman and O'Sullivan's experts weren't all that good, and there aren't that many of them to begin with.  However, detection of deception is better if the receiver has had some prior exposure to the sender -- that is, prior to the experimental test.
	A very provocative finding is that third parties, observing the interaction between the sender and the receiver, are better at detecting deception than the receivers are.

Another problem is that people do not have very accurate knowledge about valid cues to deception -- and many of their beliefs about valid cues turn out to be wrong.  When Miron Zuckerman (1981, 1985) reviewed research on nonverbal cues to deception, he discovered that there were a number of valid cues on which we could base a judgment that a person was being deceptive.  However, there were two important aspects of his findings:

• First, none of these cues is pathognomonic of lying -- that is, always diagnostic of a lie.  Some nonverbal cues are correlated with lying, but the correlations are far from perfect.
• People frequently believe that certain cues are valid that are not; or, they believe that certain cues are more valid than they are; and they ignore some cues that are, at least to some degree, actually valid.

	Some cues are components of the deceiver's verbal behavior.
	Other cues are paralinguistic -- that have less to do with what the deceiver says than how he or she says it.
	Still other cues are visual rather than vocal or auditory.
	And then there are some miscellaneous cues to deception.

Despite the availability of such cues, people are surprisingly poor at reading them -- partly because they're attending to cues that are, in fact, invalid! 

Similar findings were obtained in a review by DePaulo et al (2003), who examined more than 100 studies and more than 150 possible cues.

Presumably, though, people could be taught to read these cues properly, in the same way that Ekman's FACS system presumably teaches people to read people's emotional expressions more accurately.  If so, the detection of deception from verbal and nonverbal cues, like the reading of facial expressions of emotion, is a perceptual skill that can be acquired through perceptual learning, much as people can learn to adjust to viewing the world through distorting prisms.

This is, in fact, the premise of a program, initiated by the Transportation Security Administration, called SPOT -- for Screening of Passengers by Observational Techniques. Beginning in 2007, the TSA spent approximately $200 million per year training personnel to spot behavioral cues to deception in airline passengers' facial expressions and other aspects of body language.  However, a November 2013 evaluation by the Government Accountability Office recommended that the SPOT program be terminated, on the grounds that it was adequately supported by scientific evidence.  In large part, the GAO based its conclusions on Bond and DePaulo's 2006 review.  Ekman's response is that B&DeP relied too much on laboratory studies of "low-stakes" lies, which may not generalize to the real-world problems faced by TSA screeners.  (For a journalistic account of the debate, see"The Liar's 'Tell'" by Christopher Shea, Chronicle of Higher Education, 10/17/2014.)

Lie Detection in Forensic Settings

Ekman's studies suggest that law-enforcement personnel -- or, at least, some of them -- tend to have acquired a particular perceptual skill of lie-detection.  But Bond and DePaulo's studies, among others, suggest that most people don't have the knack.  And other critiques suggest that even the skills of trained law-enforcement personnel may be exaggerated.

First, let's get one thing -- actually, two closely related things - -straight.  Lie detectors don't work very well either.

• The traditional polygraph, which records various indices of autonomic nervous system functioning, such as heart rate and blood pressure, are very poor except under very tightly controlled conditions -- conditions that don't usually obtain in actual field settings.
• 
  
• More recent innovations, such as the use of EEG and brain-imaging methods (including so-called "brain fingerprinting") don't work any better.

The only physiological technique that is really good at detecting lies is the Guilty Knowledge Test, devised by David T. Lykken, which assesses a suspect's "secret knowledge" of the details of a crime.  So, for example, in the case of a stolen watch, the investigator might ask the suspect whether the stolen watch was a Bulova or a Rolex. All things being equal, an innocent person will not respond differentially to the two probes; but a guilty person will know the truth, and this knowledge will show up in his physiological response.  Studies have shown that the GKT produces a hit rate of 80-90%, and a false alarm rate of less than 10%.  The trick, of course, is that there has to be some aspect of the crime that only the perpetrator would know.  For this reason, it is not always possible to use the GKT; but when it's possible, it's dynamite. 

Traditionally, the GKT is performed with a traditional polygraph.  More recently, EEG and fMRI have been touted for this purpose, but in this case, the neural signature is just another physiological response.  People may be more inclined to believe a "neural signature", but it should be clear that the EEG or fMRI is nothing more than a hopped-up polygraph.

So how do law-enforcement personnel determine who is lying to them? To some extent, they rely on nonverbal cues, such as facial expressions and posture, a la Ekman.  But new work focuses on what people actually say, rather than how they say it (see "Judging Honesty by Words, not Fidgets" by Benedict Carey, New York Times, 05/12/2009).  Problems with false confessions have led police to focus their interrogations on gathering information about the crime and a suspect, instead of forcing a confession.  

But even in the determination of honesty and dishonesty, there are linguistic as well as paralinguistic cues that can make a difference.

• Liars tend to prepare a script ahead of time that is tight, but lacking in detail -- and then they stick to it.
• Truth-tellers tend to recall lots of incidental, extraneous details, they may change their stories over time, and they may make mistakes.  When recounting the event repeatedly, truth-tellers may add up to 20-30% more details over trials -- again, these are often often extraneous details.

But even these kinds of clues are far from infallible.  In one experiment, the rate of lie detection was only about 70%

In the absence of disciplined perceptual learning, however, most of us are pretty bad at detecting deception -- we perceive people as lying who are telling the truth, and we perceive people as telling the truth who are in fact lying. 

"Gaydar" as Person Perception

Person perception is the perception of a person's internal mental states of knowledge and belief, feeling and desire.  In addition to making judgments of competence, neuroticism, extraversion, and the like, we also make judgments of other people's sexuality -- both sexual orientation in general, and -- if we're interested -- sexual interest in us.  The problem of judging sexual orientation is known colloquially as gaydar -- the idea that people, especially gay people, can intuitively tell whether another person is gay or not. 

Beginning with a set of studies by Rule and Ambady (2008; Rule et al., 2009), a number of studies have demonstrated that people can identify, at better than chance levels, a target's sexual orientation based on visual, auditory, and even olfactory (don't ask) cues, even when the stimulus is severely degraded (e.g., exposures of only 50 milliseconds in duration). 

A representative study is one by Lyons et al. (2014a), in which women, self-identified as straight or lesbian, viewed head-shots of men and women who were self-identified as gay or straight on social media.  The study was conducted over the internet, and the subjects were simply asked to classify each target as homosexual or heterosexual.  Women were pretty good at this, averaging about 61% hits (i.e., classifying as gay people who really were gay, and straights as straight), and about 27% false alarms, for both male and female targets.  Both values differ significantly from the chance level of 50%.  Applying signal-detection theory yields substantial values for the d' measure of accuracy; it also revealed a bias toward classifying women as gay, especially by perceivers who themselves were lesbians.

Earlier research by Joshua Tabak and Vivian Zayas (2012) employed more degraded stimulus materials.  They presented (mostly female) judges with very brief (50 msec) flashes of faces of male and female targets who were self-identified as heterosexual or homosexual, and found that subjects were accurate in judging the targets' sexual orientation about 60% of the time -- as compared to the 50% accuracy that would be expected just by chance.  Women were more accurate than men, Judgments of women's faces were more accurate (64%) than those of men's faces (57%).  Although researchers have not (yet) uncovered the specific cues that perceivers use in this task, Tabak and Zayas found that judgments were more accurate when the faces were presented right-side up, as opposed to upside-down.  Presenting faces upside-down disrupts disrupts facial recognition -- what is known as the face inversion effect (Valentine, 1988; Farah et al., 1995).  The face inversion effect, in turn, is commonly attributed to configural processing -- that is, people recognize faces not just by recognizing someone's nose, or eyes, or mouth as individual features, but rather by recognizing the length of the nose relative to the distance between the eyes -- you get the drift (Maurer et al., 2002).  Anyway, the superiority of rightside-up presentation indicates that it was a configuration of cues, rather than individual features that was the relevant cue.  Research by Nicholas Rule suggests that the mouth may be an important cue.  Perhaps, Tabak and Zayas speculate, perceivers judged "effeminate" male faces and "masculine" female faces (as indicated, for example, by the ratio of width to height) as more likely to belong to homosexuals.  But they didn't actually test this.

Studies of "gaydar" were taken to a new level by a study reported by Yilun Wang and Michal Kosinski (JPSP, in press 2017) that garnered considerable media attention, drawing articles in The Economist, the New Yorker, and the New York Times.  Wang and Kosinski employed 35,000 images of the faces of white men and women who had reported their sexual orientation on online dating sites (there weren't enough minority gays to permit analysis).  When they presented these images to a group of human judges (recruited through Mechanical Turk), the humans' judgments of sexual orientation were correct approximately 61% of the time for male faces and about 54% of the time for female faces -- barely better than chance, and in line with the findings of Tabak & Zayas (2012).  However, when W&K submitted the same faces to an off-the-shelf pattern-recognition program, the machine's judgments were much better: 81% correct for male faces and 71% correct for females.  If the program was given five different faces for each target, overall accuracy increased to 91%.  Apparently, two factors contribute to the increased accuracy of the machine: (1) by virtue of being a computer processing a huge database, it was able to process much more cue information than would be possible for a human perceiver; (2) it employed available cue information more reliably in making its judgments.  At the same time, W&K make clear that the machine was, essentially, doing what the human judges were doing: assigning stereotypically "feminine" male faces and stereotypically "masculine" female faces to the "gay" category.  Emphasis on stereotyping.  The machine is not even, necessarily, a good model of human "gaydar", because it's likely that people rely on other aspects of appearance and behavior to make these judgments -- a man who has an inordinate interest in musical theater, perhaps, or a woman who's really into carpentry.  Of course, these too are stereotypes.  It's stereotypes all the way down.  And not necessarily accurate stereotypes, either. 

A similar problem crops up in the Ekman studies. 

• As already noted, his targets were already an unrepresentative sample of liars who were known to "leak" facial cues to deception. 
  
• Only about 1/3 of his liars were "leakers", meaning that most liar don't leak. 
  
• And meaning that the facial cues leaked by these leaky liars lose validity when applied to the population of liars as a whole. 
  
• But again, in the Ekman study, half of the targets were liars and half were not, which -- as with the homosexuals in the Lyons study, probably overestimates the proportion of liars in the population.  Perhaps not, of course, depending on the population! 
  
• But consider the problem of an airport TSA screener, who is asking each passenger, essentially, "Are you going to blow up this plane?".  The baserate is going to be low, certainly lower than 50%.  If the baserate is 10% -- on September 11, 2001, 4 of the 37 passengers on Flight 93 were hijackers -- then the true level of accuracy is going to go down from the best accuracy rate of 70% or so obtained by Ekman et al. (1999) from their sample of sheriffs and federal officers.
• Even more so when you consider that, in 2001, Newark Airport handled more than 30 million passengers, or about 2,500,000 passengers per month, or about 83,000 per day -- which means that the ratio of lying hijackers on September 11 was about 1:21,000!

Again In the current context, the accuracy of gaydar can be reformulated as follows:

So now let's return to the question of lie detection, and apply Bayes' Theorem to that situation.  To begin with, here's an example from the Conceptual Tools website developed by Neil Cotter, a professor of electrical engineering at the University of Utah.  In his discussion of Bayes' Theorem, he considers a polygraph lie detector that detects lies with about 90% accuracy.  That is, the probability that the lie detector says "You lied" when you really did lie is .89, and the probability that the machine says "You told the truth" when you really did tell the truth is .90.

Applying Bayes' Theorem, p(L | DL) = ((.89)*(.05)) / (((.89)*(.05)) + ((.10)*(.95))) = .32.  In other words, what looked like an accuracy rate of 9/10 is reduced to about 1/3.

Now what about human lie-detection?

Let's consider first the Bond & DePaulo (2006) study, which found an overall accuracy of 54%.  But those studies employed targets who were 50% liars and 50% truthtellers.  If we follow Cotter's example, above, and assume that the base rate of lying is 10% (not his 5%), the probability of correctly identifying a liar drops to p = .12 -- a substantial loss of predictive accuracy.

Now let's turn our attention to Ekman et al.'s (1999) study of lie detection.  Recall that their best human lie-detectors, most of whom were CIA agents, were 80% accurate in detecting liars.  They were 66% accurate in detecting truthful statements, which means that they called truth-tellers liars 34% of the time.But that was with 50% liars in the target pool.  Suppose that the proportion of liars is smaller than that -- say, 10%.  Applying Bayes' Theorem, the accuracy of these Federal Officers drops considerably, to p = .21. 

Figure and Ground in Person Perception

For a variety of reasons, most work on nonverbal aspects of person perception is based on the assumption that certain physical stimuli are intrinsically related to certain mental states.  One example is Ekman's work on facial expressions of emotion; another is Zebrowitz's work on babyfacedness.  But one of the important lessons of nonsocial perception is that stimulation is ambiguous, forcing perceivers to go "beyond the information given" by the stimulus to form mental representations of the external world.

• One way of going "beyond the information given" is to draw on pre-existing knowledge, expectations, and beliefs.
• Another way is to pay attention to the context in which the stimulus occurs. 

"Which photo do you believe?  Photograph #1... was shot by Alan Diaz of the Associated Press.  Were other shots taken in the instants before and after?  Was anything posed?  Who was first to grab the child?  What were the subjects saying.  The news organization is objective; we'll believe its report.

 

"Photograph #2... was credited "courtesy of Juan Miguel Gonzalez," carefully posed for propaganda purposes.  It was taken -- after nobody knows how much cajoling -- by Gregory Craig, President Clinton's personal lawyer, who was hired by [a] left-wing church group serving Fidel Castro's interests"

Stimulus and Context in "Academic" Art

Up through the 19th century, so-called "academic" artists - -that is, artists who had been schooled in one of the Academies of Fine Arts, as opposed to those who had been apprenticed to a master -- were taught formal rules for portraying various emotions through certain facial expressions, postures, and gestures.  These artists followed these rules, so that viewers would understand what their paintings were attempting to convey.  However, many artists discovered that they could use these same expressions, postures, and gestures to convey precisely the opposite emotion, depending on the context.  They could depend on the fact that their viewers shared the same knowledge and expectations as the painter, and so would interpret the painting properly.  

In an example made famous by the art historian Edgar Wind (1937, 1986), the 18th-century English portrait painter Joshua Reynolds (who became the first president of the British Royal Academy) noted that  

"There is a figure of a Bacchante [also known as a Maenad) leaning backward, her head thrown quite behind her, which... is intended to express an enthusiastick frantick ([both sic] kind of joy....  This  figure Baccio Bandinelli, in a drawing... of the Descent from the Cross, has adopted... for one of the Marys, to express frantick agony of grief.

The Bacchante	The Cross	Comparison

Here is another example, also from Reynolds via Wind 

Another Bacchante	Another Cross	Another Comparison

The same posture on the body, the same gesture of the arms, the same expression on the face -- these convey frantic joy and enthusiasm when presented in the context of the drunken, licentious, orgiastic revels associated with the Roman god Bacchus (also known as the Greek god Dionysus, from which we get the word bacchanalia), the Greek god of wine; but they convey frantic grief and agony when presented in the context of the crucifixion and death of Jesus, whom Christians believe to be the Word of God incarnate.  The difference in interpretation is created by differences in which the context is presented.  

Other art historians have noted similar instances of the contextual reversal of emotional meaning.  

For example, E.H. Gombrich (who himself wrote many interesting books about the psychology of art) reports that Aby Warburg (1866-1929), a German historian of Renaissance art (the Warburg Institute in London is named after him), took a great interest in such context-based "inversions" of meaning.  

	"The artist who uses [classical 'engrams', or symbolic poses]... can use them in a different context, 'invert' their original savage meaning, and yet benefit from their value as expressive formulae.  In this way, Bertoldo de Giovanni had used the model of a pagan maenad to give expression to the passionate grief of the Magdalen under the cross,
	and Donatello... used a relief on the cover of a sarcophagus, in which Pentheus is torn to pieces by maenads, for a composition of his own in the Santo in Padua.  Where pagan frenzy had represented a leg torn off by insensate women, the Christian artist 'inverted' the scheme to glorify the healing of a broken leg.
	Similarly Agostino di Duccio's relief representing the rescue of two children by San Berbardino adapts the forms but ''inverts' the meaning of a classical sarcophagus which represents the terrible tale of Medea's  murder of her children."

Wind notes: "Perhaps the shrewdest advice Sir Joshua Reynolds gave his students was... a fundamental law of human expression: 

'It is curious to observe, and it is certainly true, that the extremes of contrary passions are with very little variation expressed by the same action'".

This is possible because context changes the perception of the stimulus; or, put another way, the stimulus varies from context to context.

The moral of the story is that in order to properly perceive an object or event, including -- especially -- a social object or event, we must extract information from the stimulus in context, because the context is also part of this stimulus, and combine information extracted from the stimulus-in-context with pre-existing knowledge, expectations, and beliefs stored in memory.  

Perception involves extracting information from the stimulus.  But as F.C. Bartlett (1932) forcefully reminded us, "The psychologist, of all people, must not stand in awe of the stimulus".  As Jerome Bruner has argued, perception entails "going beyond the information given" by the stimulus, combining information from the stimulus, and its environmental context, with other knowledge, beliefs, and expectations -- what might be called the cognitive context of perception.  

Perceiving Behavior

The study of social perception is dominated by the problem of person perception, or impression formation, but we don't just perceive people: we also perceive their actions.

Wegner and Vallacher proposed their action identification theory to describe what goes on when we think about their own actions, and the actions of others that they observe.  In particular, they were interested in how we focus on low-level details or high-level gist, and on how the meanings of events change as we get closer to them, or further away, in time.

The Accuracy of Person Perception

One implication of the self-fulfilling prophecy, and related effects, is that it doesn't really matter whether an actor's perceptions, expectations, and beliefs are correct -- all that matters is what they are, because those internal mental states determine our behavior.  But obviously, accuracy is important.  The purpose of perception is to enable us to know the world around us.  This raises the question of the accuracy of social perception: do the mental representations of the objects and events we encounter in the social world accurately reflect their actual existence, structure, and states?  The question of accuracy arises in nonsocial perception as well, as exemplified by research on visual and other illusions.  The Gibsonian approach assumes that the perceptual apparatus evolved in such a way as to enable us to perceive the world the way it really is.  But the constructivist approach admits that even our nonsocial percepts can be biased and distorted by knowledge, beliefs, and expectations.  This must be even more the case in the social domain, given the vague, fragmentary, and ambiguous nature of the stimuli we encounter in the social world. 

As Kenny and Albright (1987) note, interest in the accuracy of person perception has its roots in the intelligence-testing movement in the early 20th century.  If psychologists could measure individual differences in intellectual skills, then they ought to be able to measure individual differences in social skills as well.  Chief among these is empathy, which we can define as a person's ability to understand the attitudes, feelings, and experiences of another person.  And, as a matter of sheer logic, empathy requires accuracy in social perception -- the ability to accurately read another person's mental states.  Accuracy was also an issue in the analysis of clinical decision-making -- that is, whether psychiatrists and clinical psychologists were accurate in diagnosis mental illness, or predicting the outcome of treatment.  Not to mention personnel selection, including college admissions: is an applicant the "right person" for a particular job or school?

A major feature of this early work was a debate over clinical vs. statistical prediction -- that is whether clinicians' impressions of patients, derived from their subjective appraisal of the patients themselves, interview records, and psychological testing, was superior to "actuarial" predictions derived by techniques such as multiple regression from objective data.  The answer, quite clearly, was no.  This was the finding of early studies by Sarbin (1943) and Meehl (1954), and has been born out by virtually every study since then (e.g., Mischel, 1968; Wiggins, 1973).

• No matter what the assessment context, statistical methods of combining information have proved to be generally superior to impressionistic "clinical" methods.
• The vanishingly few exceptions to this rule, which find that clinical prediction is superior to statistical prediction, are methodologically flawed, and corrections showed, once again, that statistical prediction was superior.
• In some studies, clinical and statistical prediction were tied; but because statistical prediction is more economical (in terms of time, effort, and dollars) than clinical prediction, statistical prediction wins on grounds of utility (efficiency, cost-benefit analysis).  
  
• Regression equations based on unit weighting predict as well as more complicated ones based on actual beta weights.
• Predictions based on objective tests have more utility than those based on "projective" tests like the Rorschach or Thematic Apperception Test.
• Predictions based on the paper record have more utility than those based on interviews -- which is only one reason to oppose the way we organize Visitors' Day for potential graduate students.
• The only exception is the critical incident interview pioneered by David C. McClelland and his associates, which is tantamount to a performance-based work-sample.  But hardly anyone ever does a genuine critical incident interview, so the point is moot!

An excellent example of the virtues of statistical prediction comes from a study of objective lie detection by Hartwig & Bond (2014).  They acknowledged, based on reviews such as DePaulo et al. (2003) and Bond & DePaulo (2006), that subjective lie detection is not very good.  That is, people's subjective impressions of whether people are lying are not very accurate -- barely better than chance.  They then raised the question of the accuracy of objective lie detection.  That is, given all the various cues to deception, could a statistical algorithm combine all the available data to produce predictions of deception that would surpass the subjective, "clinical" impressions of human judges.  Of course they could.  For this purpose, H&B calculated the correlation coefficients between each of some 60 cues surveyed by DePaulo et al. (2003), and then entered these correlations into a multiple regression equation.  Although the validity of the individual cues was relatively low, M |r| = .24, the multiple R =.52. 

As a matter of statistics, such correlations are inflated by chance associations, so the proper method is to engage in double cross-validation.   That is, you divide your sample in half, and calculate R for each half separately, and apply each of the resulting regression equations to the other half of the sample -- that is, the sub-sample from which it was not derived.  The resulting cross-validity coefficient R was .42.  This is lower than .52, to be sure, but it's still pretty good.  The cross-validated multiple-regression equation yielded a validity of 68%, a substantial improvement over the 54% seen in the subjective, impressionistic, "clinical" judgments.  The multiple R were highly stable across various conditions, such as the liar's demographic background, the motivation to deceive, the deception medium (visual, oral, written), etc. 

Hartwig and Bond (2014) concluded that "signals of deception are manifested in constellations rather than single cues".  But the more basic point is that there are, after all, valid cues to deception.  The problems with subjective, "clinical" judgments of deception are:

As if the advantage of statistical over clinical prediction weren't bad enough, in 1955 Cronbach began publishing a series of papers that called into question most of the research on accuracy that had been published up to that time.  In order to understand Cronbach's critique, consider a simple impression-formation experiment in which a a group of subjects must judge each of a set of targets on a set of traits (like the Big Five).    In his analysis, Cronbach argued that both the judgments and the criteria against which they are validated consist of four components of accuracy. 

Never mind, for a moment, what the objective criterion for a trait like "extraversion" is!  Assume that the judges are basing their impressions on video clips of the targets, and that their impressions are being validated against the targets responses to an objective personality questionnaire.  Never mind, for a moment, the validity of the personality questionnaire!

• Elevation: the tendency of a judge to rate all targets favorably or unfavorably. 
  
• For example, a judge might rate all targets favorably on all the Big Five dimensions.
  
• Stereotype Accuracy:  the tendency of a judge to rate all targets more or less favorably on a particular trait, compared to other judges.
• For example, a judge might rate subjects more favorably on Extraversion than other judges do.
  
• Differential Elevation:the tendency of a judge to view some particular target more or less favorably than other judges do.
• For example, a judge might rate a particular target more favorably on all the Big Five Dimensions.
  
• Differential Accuracy: the judge's view of a particular target on a specific trait, after the other three components have been removed from the equation. 
  
• For example, a judge might rate a particular target more favorably on Extraversion than other subjects do.

Cronbach's basic point was that differential accuracy is at the heart of accuracy, because it has to do with the uniqueness of the individual target.  And it can't be evaluated until the other components of accuracy have been accounted for, and removed from consideration -- which, Cronbach argued, hardly anyone had done up to that point.

Cronbach's point was well taken, but his critique had the unintended effect of stopping research on accuracy dead in its tracks.

• Experimental social psychology shifted its focus to attitudes and persuasion, as well as the study of situational influences on behavior, as exemplified by the Milgram experiment.
• It did not help, probably, that the cognitive point of view, as exemplified by the Thomas Theorem, symbolic interactionism, and the like, implied that "accuracy" was beside the point.  What mattered was the way the actor perceived the situation, not whether the perception was accurate.
• To make things even worse, beginning in the late 1970s and early 1980s the "errors and biases" program in social cognition (initiated in response to the early work of Kahneman and Tversky, among others) simply assumed that social perception was inaccurate, riddled with errors (like the Fundamental Attribution Error) and biases (like the self-serving bias in causal attribution). 

Still, interest in accuracy did not disappear entirely.  It was maintained by researchers in judgment and decision-making, who continued to be interested in the accuracy -- or, at least, the adaptiveness -- of social judgment.  I'll have more to say about this line of research and theory in what follows.  It was also maintained by a new generation of personality researchers, who insisted -- against the claims of social psychologists -- that personality traits really did exist, they were not merely figments of the imagination, and could be judged on the basis of behavior by external observers and targets themselves.  Given the assumption that personality traits really did exist after all, it made sense to consider how accurate judgments of personality were.  Hence, a new marriage was consummated, between personality and social psychologists, around the topic of person perception -- of impressions of personality and their validity.

Note. Much of the following discussion is heavily influenced by a paper by Reid Hastie and Kenneth A. Rasinski, "The Concept of Accuracy in Social Judgment", which appeared in the Social Psychology of Knowledge, edited by D. Bar-Tal and A. Kruglanski (1988).  See also the very useful discussion by Hastie and Robyn Dawes in Rational Choice in an Uncertain World (2001; 2nd ed., 2010).

The first problem for social perception is the same as in the nonsocial  domain: what information (if you will, the proximal stimulus) is displayed by the target (the distal stimulus), and how does stimulus information combine with the perceiver's pre-existing fund of knowledge and schemata to yield a perception of the person's internal mental state -- regardless of whether that perception is accurate? This is the problem addressed by Brunswik's lens model of perception.

The classic approach to accuracy in perception comes to us from Egon Brunswik (1947), a Hungarian psychologist who established the first psychological laboratory in Turkey (at the University of Ankara), but who taught for most of his career at UC Berkeley and was a close associate of E.C. Tolman.  Brunswik's theoretical point of view,  known as probabilistic functionalism, is best expressed in his monograph on Perception and the Representative Design of Psychological Experiments (1947; 2nd edition, 1956; see also Hammond, 1966, 1998).

Brunswik based his analysis of perception on his lens model, which argues that the individual perceives the world through a "lens" of imperfect cues (the diagram representing the model also looks a little like a lens).  Recall that the goal of perception is to form an internal mental representation (what Brunswik called the achievement) of the distal stimulus.  In this sense, "accuracy" may be defined in terms of the match between the features present in the stimulus and those cues present in the percept.  Brunswik's model was quickly generalized to the realm of judgment and decision-making, which is where the lens model has been most frequently applied.  For purposes of the present discussion, we can consider the perception of a person, or our impression of a person, as tantamount to a judgment concerning his internal mental states (beliefs, feelings, desires) and personality -- whether he's happy or angry, neurotic or extraverted.

Here is the basic vocabulary of the lens model, as applied to judgment and decision-making.

• The distal stimulus, or the target's actual internal states and traits, is called the distal stimulus, inferred state, or -- simply -- criterion.
• In nonsocial perception, these states are physical features such as whether the object is near or distant, stable or in motion, rigid or flexible.
• In social perception, these states are psychological features such as whether the person is extraverted or introverted, happy or sad, gay or straight, lying or truthful.
• The percept or mental representation of the target's states and traits is called the response, subject's inference, or -- simply --  judgment.
• The relationship between the judgment and criterion is called the achievement, correspondence relationship, or inference.
• Accuracy, then, reflects the degree of correspondence between the criterion and the judgment.
• The distal stimulus presents proximal cues which form the basis for the judgment.
• These cues vary in terms of their ecological validity, or the strength of correlation between the distal stimulus and the proximal cues available to the perceiver.
• The cues themselves have objective values.
• And because the cues are correlated with each other, there are objective inter-correlations among them.
  
• The cues also vary in their cue utilization, or the extent to which they play a role in the subject's judgment.
• On the subject's side, the utilized cues have a subjective value in terms of the inference he is trying to make.
• These subjective values may be invalid.
  
• And, in addition, we have to consider the subject's beliefs the subjective inter-correlations among the cues.
• It goes without saying that these beliefs may also be invalid.
  

• People base judgments of masculinity and femininity on certain facial features, such as length and width, sharp vs. rounded corners, shape of the eyes, eyebrow thickness, eyelash length, sharpness of cheekbones, and the like.
  
• And they judge sexual orientation by whether a face is sex-atypical -- i.e., a masculine face on a woman or a feminine face on a man.
  

Obviously, though, this says nothing about the accuracy of these judgments, or the validity of the cues they're based on.  And, as you might expect, there are many slips between the cup and the lip.

• The distal stimulus may not make ecologically valid cues available for perception.  If the cues are ecologically invalid.
• A good example of this in the nonsocial case is the Ames Room, which is specifically constructed to provide incorrect cues to the observer.
• In the social case, recall that about 2/3 of Ekman's targets did not "leak" valid cues to deception. 
• Cue utilization validity may be discrepant from ecological validity. 
  
• The observer may not utilize ecologically valid cues available in the stimulus environment.
• Even with ecologically valid cues, the subjective values may differ from the objective values.
• And their subjective inter-correlations may differ from their objective intercorrelations.