Let me give you a simple example, taken from the Centennial Exhibition, "Understanding Ourselves, Understanding Each Other", sponsored by the American Psychological Association. At some point in the exhibit, you come upon a corridor paved in a black and white checkerboard pattern, with a sign -- multilingual, yet -- warning you to step only on the black squares. Everybody does this, of course, and when you get to the other end you are given a little lecture on mindless conformity. Except, of course, there's nothing mindless about it. More than likely, the average visitor went beyond the information given in the sign, inferred that there must be a good reason for the injunction, and decided to behave in a cooperative manner. To use a real-world analogy, we hardly ever demand to know why we cannot cross police lines; mostly we figure that there must be a good reason for the barrier (perhaps there's a hole in the sidewalk; perhaps there's evidence that needs to be protected). Neither case is fairly construed as mindless obedience to authority; they might well reflect reasoned, cooperative behavior, the equivalent of simply passing the salt if asked whether you could.

Of course, you don't need to be a Gricean sociolinguist to think about experiments that way. Martin Orne had the same kind of idea in his arguments, which he began to voice in the late 1950s and early 1960s, about demand characteristics and the ecological validity of psychological experiments. Over the years, both notions -- demand characteristics and ecological validity -- have been somewhat controversial, and I thought I would take this opportunity to remind people what the argument was all about, and reflect on its connection to Gricean sociolinguistics, and on the meaning of both for what we do as psychological scientists.

From Orne's point of view, the purpose of laboratory research is to understand the real world: to make the problem simple so that it can be studied effectively; and to control relevant variables so that important relations, especially causal relations, can be revealed. Unfortunately, generalization from the lab to the real world requires an inferential leap: its legitimacy depends on the degree of similarity between the conditions which obtain in the laboratory and those found in the real world. In the natural sciences, perhaps, it is safe to assume that the lab is in important respects like life. But in psychology, Orne argued that this assumption is not safe.

The situation is bad enough in animal research -- for example, Martha McClintock's studies of sexual behavior in rats show how theory can be greatly misled when the experimental situation is not naturalistic. But it is even worse in human research, for the simple reason that human subjects are not reagents in test tubes, passive responders to the experimenter's manipulations. They are sentient, curious creatures, constantly thinking about what is happening to them, evaluating the proceedings, figuring out what they are supposed to do, and planning their responses. These normal human cognitive activities may interact with experimental procedures in unknown ways. At best, they can obscure the effects of a manipulation, by adding noise to the system; at worst, they can render an entire experiment invalid.

For Orne, the experiment is a unique form of social encounter, with roles and rules that are not found anywhere else (except perhaps in doctors' offices). This uniqueness may preclude generalizations from lab to life -- which is what Orne means by threats to ecological validity. In the first place, the human subject is a volunteer who, in addition to his or her desire for remuneration (whether in the form of cash or research participation points) has an emotional investment in research which stems from three goals: to further scientific knowledge; to find out something about him- or herself; and to present him- or herself in a positive light. Second, the experimenter and the subject enter into an implicit contractual arrangement with specifically defined roles: the subject agrees to tolerate deception, discomfort, and boredom for the good of science; and the experimenter guarantees the safety and well-being of the subject (this guarantee legitimizes any requests that the experimenter might make).

As an aside, let me point out that in terms of this analysis, the injunction of the 4th edition of the APA Publication Manual, that we substitute "participants" for "subjects" when talking about human beings, may be politically correct but it is not psychologically correct. Both the experimenter and the subject are participants in the social encounter known as the experiment, and each has his or her own unique role to play in that encounter. The experimenter is the experimenter, and the subject is the subject.

Third, the experiment is by its very nature episodic: in important respects it is divorced from the rest of the subject's life experiences; and, in any event, it is expressly time-limited, and should have no permanent consequences for the subject. Fourth, subjects perceive the experiment in the context of their entire experience. They are engaged in "effort after meaning", trying to discern what the experiment is all about and deciding what to do. In so doing, they pick up on what Orne called the demand characteristics of the experiment, by which Orne meant the totality of cues available in the experimental situation which communicate the experimenter's design, hypotheses, and predictions. Some of these cues are explicitly present in the experimenter's instructions to the subject, but many of them are implicit in the solicitation materials, campus scuttlebutt, incidental remarks made by the research assistants, and hints communicated by the procedures. The important thing to understand is that demand characteristics aren't just communicated by the experimenter, and unlike experimenter bias, they can't be controlled by keeping the experimenter blind to the experimental hypothesis. Rather, some demand characteristics are brought into the experiment by the subject, while others arise as the experiment proceeds; in either case, they're everywhere. In the final analysis, they are internal to the subject, they can't be predicted in advance by someone external to the experiment, and in principle they cannot be controlled; they can only be evaluated.

The point is that regardless of the experimenter's intentions and instructions, the subject's behavior in the experiment is determined by his or her perceptions of the experimental situation, perceptions which are formed as the subject goes beyond the information given by the experimenter. For this reason, the subject's perceptions may be at variance with the experimenter's intentions. If this occurs, the experimenter and the subject are literally participating in two different experiments, and ecological validity is lost.

An anecdote from the early days of sleep research: One night a subject arrived for an experiment which was, truthfully, advertised as recording physiological responses while subjects were asleep. One participant dutifully submitted to having electrodes attached to his head and other body parts and crawled into bed; the experimenter told him to go to sleep and turned out the light. Half an hour later, the subject was still awake. The experimenter asked if everything was all right, the subject replied yes, experimenter reminded the subject to go to sleep, and the subject agreed. Half an hour later the subject was still awake, the scene was repeated, and again half an hour after that, and again half an hour after that. Finally, the experimenter burst into the bedroom in a fit of exasperation and demanded to know why the subject was refusing to go to sleep. To which the subject replied, "You mean the mouse in my bed isn't supposed to be there?"  The story may be apocryphal, but the experience is ubiquitous. The very first group hypnosis session I ran at Wisconsin, with Ernie Mross and Paula Niedenthal, both now PhD psychologists, as novice undergraduate research assistants, a subject became slightly nauseated just as the tape-recorded induction procedure began -- not from the hypnosis, mind you, but from a particularly potent combination of beef stroganoff and Old Milwaukee beer which he had just eaten at the student union. We escorted the subject out of the room under the watchful gaze of about 125 other subjects, and the rest of the session went smoothly, but we had a devil of a time convincing many subjects that the whole thing hadn't been staged to see how their responses would be affected. Paula, Ernie, and I were just trying to assess hypnotizability; these subjects thought we were doing something else entirely.

This happens all the time in personality research, where subjects can't believe that all you want them to do is fill out a questionnaire. Once they've read about the bystander intervention studies, they keep expecting smoke to pour in through the ventilation ducts, or for a research assistant to fall and break his foot.

So in order to make sense of experimental outcomes, the experimenter must attempt to understand the subject's behavior from the subject's point of view. Unfortunately, this understanding is impeded by what Orne called the pact of ignorance implied by the experimental contract. Both parties want the experiment to work; therefore, the subject agrees not to tell the experimenter that he or she has figured out the experiment, while the experimenter agrees not to force the subject to admit that he or she possesses this forbidden information. The situation is caricatured by a scenario in which the experimenter, who has already mentioned to the subject that the experiment is part of her dissertation research, pulls the subject's payment out of her purse, puts it on the table in front of the subject, debriefs the subject as to the actual purpose of the experiment, and then asks one last question: Did the subject catch onto any of this? The subject dutifully replies "no" (else he would have just wasted both the experimenter's time and his own), the experimenter breathes a sign of relief, hands over the money, and they both go on their respective ways.

In order to break the pact of ignorance, Orne argued, the experimenter and subject must alter their usual roles, concluding the experimental episode and transforming what once was a subject into a genuine co-investigator, who feels it legitimate to reflect truthfully and dispassionately on what has gone on before. That's what Orne's real-simulator design was all about. Simulators aren't subjects in the usual sense, because they are only pretending to be in an experiment. They're not controlling for demand characteristics, or indeed for any other experimental variable; they are collaborators of the experimenter, helping to evaluate the experimental design.

First, an experiment on whether antisocial and self-injurious behavior can be induced by hypnosis. This question goes back more than 200 years, to the French Royal Commission's investigation of Franz Anton Mesmer, and more recently was even asked by the Central Intelligence Agency. And it's a legitimate one: hypnotized subjects are highly responsive to suggestions, especially for perceptual distortions, and perhaps this suggestibility gives the hypnotist a special power to coerce antisocial and self-injurious behavior.

In 1939, Rowland had reported an experiment in which subjects were hypnotized, placed in front of a large, active, diamondback rattlesnake, told it was a coil of rope, and asked to pick it up. One of two subjects who received this request immediately complied, at which point he struck his hand on a pane of invisible glass which had been interposed between him and the reptile. By contrast, 41/42 control subjects, asked if they would do the same thing, refused to go anywhere near the snake. In 1952, P.C. Young, who had been one of Hull's students at Yale, replicated Rowland's finding: 7 out of 8 subjects attempted to pick up the snake and threw a vial of nitric acid at a research assistant (who was also protected by invisible glass).

Orne and Evans were deeply suspicious of both experiments, because their procedures, appeared to violate the basic contract between experimenter and subject: the subject agrees to do what the experimenter wants, and the experimenter agrees to protect the subject from harm. The request to pick up the snake or throw the acid has some of the character of "Could you pass the salt?": the subjects go beyond what the words mean to infer what the experimenter means. It's not enough that 41/42 unhypnotized controls said they wouldn't pick up the snake: they might behave quite differently in the actual experimental situation.

Anyway, Orne and Evans contrived an experiment in which a group of highly hypnotizable subjects were hypnotized and asked to reach into a wooden box for all sorts of things: a 2-shilling coin (this experiment was done in Australia), a piece of chalk, a harmless gecko lizard, and a harmless green tree snake. All the subjects did everything, except one subject who fainted at the sight of the snake. The surviving subjects were then asked to pick up a new snake. This was a red-bellied black snake, otherwise known as the Australian two-step, because that's how far you get after it's bitten you. All five of the subjects attempted to pick up the snake, which of course was shielded behind a pane of invisible glass. They also were willing to remove a partially dissolved 2-shilling coin from a beaker of fuming nitric acid (this is perfectly safe if you do it right, but don't try it at home), and finally to throw the acid at poor Fred Evans, who was also protected by invisible glass. All of the subjects eagerly complied. But so did a group of insusceptible subjects who had been instructed to simulate hypnosis; and, for that matter, so did a group of unselected subjects run in the normal waking state. All of them, interviewed later, reported that they felt perfectly safe in the experiment, secure in the knowledge that appropriate safeguards were in place -- as in fact they were. Their safety had been clearly communicated by the demand characteristics of the experiment.

Reading about such an experiment makes you re-evaluate the procedures used in Milgram's classic studies of obedience to authority. As you'll remember, Milgram ostensibly recruited subjects in pairs, and, ostensibly, randomly assigned one to be the teacher and the other the learner in an experiment on punishment and learning. To make a long story short, the learner was a confederate of the experimenter, making errors according to a prearranged script, and a surprisingly large number of subjects were willing to administer intense punishment in response to the learner's apparent mistakes. To an external observer, the level of obedience is chillingly compelling. But then you have to ask yourself some questions from the subject's point of view -- chiefly, "What am I doing here?"  If my only job is to administer punishment, why can't the experimenter do it himself, and run both of us as learners? If he's interested in the effects of punishment on learning, why is he in here watching me?@ The totality of cues present in the situation -- even when it's conducted in a run-down building in Bridgeport rather than the hallowed spires of Yale -- are enough to lead the teacher to conclude that things are not what they appear to be, and to generate the hypothesis that he, not the learner, is the actual subject of the experiment. If so, the deception has failed, the experimenter and the subject are in different experiments, and all bets are off.

I know that the Milgram experiment is a classic (in fact, that's what the black-and-white checkerboard is all about at the APA exhibit), and I'm not saying that it was nothing but demand characteristics. Milgram might (in fact, he did) say that the power of the experimental situation was an illustration of his points about the power of situations in general. Perhaps; but, as Orne noted at the outset, generalization from the lab to life depends on the experimenter and the subject being in the same experiment. This dispute won't get resolved anytime soon, so let me be clear that I'm just using the Milgram experiment as a familiar example of how, in order to understand experimental outcomes, we have to understand experimental procedures from the subject's point of view.

The argument from demand characteristics got a reputation as a spoiler of experiments, but that's because most people who made the argument failed to appreciate what it was all about. You don't use demand characteristics to discount experimental outcomes; you evaluate them to make your experiments better, more ecologically valid, more convincing.

A nice example of this principle comes from another Orne and Evans gem, "The Case of the Disappearing Hypnotist". Hypnosis is commonly construed as something that the hypnotist does to the subject. This isn't right, but it is a common impression, and it occasionally leads to questions from subjects like "What happens if you drop dead while I'm hypnotized?". Roy Dorcus, a hypnosis researcher at UCLA, contrived an experiment to find out. While in the middle of a hypnosis session, he arranged to be interrupted by his secretary, who loudly reminded him that he had an appointment downtown, at which point Dorcus got up and left the room, without a word to the subject, never to return. Hypnotized subjects remained in the room for an average of 28 minutes, compared to 23 minutes for a relaxation control group. The difference wasn't significant, but in the final analysis it's not clear what conclusions one should draw from the results. This is because the experimenter's unannounced departure violates the experimental contract, and may have been perceived to be part of the experiment.

Orne and Evans performed an elaborate conceptual replication of Dorcus's study, with hypnotic subjects and simulators. The subjects were recruited for a study of the physiology of hypnosis, and when they arrived at the laboratory they were connected to a polygraph, hypnotized by a tape-recorded procedure, and administered a series of suggestions while a research assistant observed their responses. In the middle of one of these suggestions, while the subject's eyes were shut, one arm stuck to the chair, tapping his foot in time to hallucinated music, another assistant threw a silent switch, the desk light went out, the tape slurred to a halt, the assistant muttered "The damn fuse", opened the door, noisily ran down the hall, and never returned. Actually, she tiptoed back into an adjacent room, where she observed the subject through a one-way screen, and reappeared in the experimental room 30 minutes later. The results were striking: on each of four dependent variables, the simulators outlasted the reals. They kept their eyes closed for an average of 25 minutes, continued tapping for 17, kept their arms fixed to the chair for 23; and when the assistant returned to the room, five out of six appeared to be deeply hypnotized. A postexperimental interview clarified what had happened. The room was outfitted with a conspicuous mirror, which was in fact a one-way screen; and while the desk lamp and tape went off, the polygraph and a ceiling light stayed on (some light was required for observing the subject, and the experimenters assumed that subjects would realize that different outlets in the same room might be on different circuits). The simulators clearly smelled a rat, and so they behaved the way they supposed a hypnotized subject would actually behave.

"Oh, what a tangled web we weave

When first we practise to deceive." 

This was the epigram, from Sir Walter Scott's Marmion (1808), to Orne's critique of the Milgram experiment. Orne was very suspicious of experimental deceptions and cover stories, and his critique has become conventional wisdom for many social psychologists, who often use deception in their experiments. (This reliance on deception is why social psychology has traditionally been discussed at the end, but not the beginning, of the introductory course: by the time students learn about experimental deceptions, the social psychologists have already gathered their data.)

By contrast, cognitive psychologists haven't worried about this problem all that much. Reid Hastie, asked what made him a social psychologist and not a cognitive psychologist, replied that he lied to his subjects. But it's not true that cognitive psychologists don't deceive to their subjects. Consider the levels-of-processing paradigm in memory research, in which subjects are told that the experiment is about how people make judgments about words, when in fact they are going to be surprised with a memory test. Consider research on explicit and implicit memory, in which experimenters go to great lengths to convince their subjects that the stem-completion test (for example) has nothing to do with the wordlist that the subject studied only moments before. In the final analysis, the problem of demand characteristics isn't just attached to deception experiments. Because every subject is engaged in figuring out the meaning of every experimental situation, demand characteristics are a problem.

And because experimenters and their subjects are always engaged in a conversation, the logic of conversations is also an enduring problem. Consider work on judgment and decision making, in which many of the problems posed to subjects violate Grice's conversational rules. When we ask subjects in a consumer preference survey to indicate which pair of stockings they prefer, given the context have every right to believe that the question is answerable -- in this case, that items are in fact different; and when asked to justify their choices, they have every reason to reject as unresponsive (if not also impolite), the fact that the chosen pair was on the right rather than the left side of the display. When we ask subjects to predict whether a person is an engineer or a lawyer and then describe that person as someone who is uninterested in politics and social issues and likes woodworking and mathematical puzzles, the subject has every right to believe that this individuating personality description is somehow relevant to their task, and to use the information somehow. And when they do so, we have no right to conclude that people are irrational or don't understand normative rules of inference. Maybe they do and maybe they don't, but in their conversation with the experimenter they are only doing what comes naturally: assuming that the experimenter is following the cooperative principle and its four associated maxims, they seek to find common ground, and generate their response from this stance. To assume otherwise, in the absence of an understanding of the experiment from the subject's point of view, is to risk serious misunderstanding of how mind and behavior work.

Orne was concerned with ecological validity, and with the peculiar character of the experimental situation. To a great extent, he thought that demand characteristics were a problem because of motives -- to help the experimenter, to learn about themselves, and to look good -- that were peculiar to research participants. Grice reminds us, though, that there is another motive which subjects display both in the lab and elsewhere in life. Subjects aren't just motivated to guess and confirm the experimenter's hypothesis; as listeners -- that is to say, as people -- they are primarily motivated to make sense of any communicative situation in which they find themselves. In that respect, at least, Orne needn't have worried, for what happens in the laboratory is entirely representative of what goes on in the real world. Because the laboratory is just like the real world after all, it follows that, in our experimental conversations with our subjects, as we establish common ground and collaborate with them in learning about the mind and behavior, we must be careful to follow Grice's maxims:

	be cooperative,
	be informative,
	be true,
	be relevant, and
	be clear.

¹Apparently this sentence first appeared in papers by Gordon and Lakoff (19xx) and Searle (1975); according to Herbert Clark (personal communication, June 26, 1995), there also exists a satirical paper entitled "Can you pass the salt?" or somesuch. The earliest psychological reference to this problem is in Clark and Lucy (19xx) and Clark (1979).  Return to text.