Demand Bias in the Assessment of Situational Effects on Buyer Behavior

ABSTRACT - The possibility that the conventional assessment of situational effects on buyer behavior introduces demand bias was investigated. Both demand awareness and subject role set were manipulated in four conditions. The results substantiated that possibility.

Citation:

Peter H. Reingen (1976) ,"Demand Bias in the Assessment of Situational Effects on Buyer Behavior", in NA - Advances in Consumer Research Volume 03, eds. Beverlee B. Anderson, Cincinnati, OH : Association for Consumer Research, Pages: 130-133.

Advances in Consumer Research Volume 3, 1976      Pages 130-133

DEMAND BIAS IN THE ASSESSMENT OF SITUATIONAL EFFECTS ON BUYER BEHAVIOR

Peter H. Reingen, Iona College

ABSTRACT -

The possibility that the conventional assessment of situational effects on buyer behavior introduces demand bias was investigated. Both demand awareness and subject role set were manipulated in four conditions. The results substantiated that possibility.

INTRODUCTION

Only recently has any systematic research been undertaken to assess the impact of situational variables on buyer behavior (Belk, 1974a,b; Belk, 1975; Lutz and Kakkar, 1975). Based on the premise that consumer behavior is a function of the interaction between the individual and the situation, this research provides evidence that substantially more behavioral variance can be accounted for by explicitly introducing situational variables into the research design rather than by ignoring them. Thus, this line of research offers much promise.

In the standard investigation of situational effects (e.g., Belk, 1974a), the subjects respond to a modified behavioral differential questionnaire in which the likelihood is indicated that they would choose each of a number of responses (products) to a stimulus configuration (situation). The result is a treatments (situations) by treatments (products) by subjects design which allows for the estimation of the relative contribution of the main and two-way interaction effects to the variance in the dependent variable. The patterns of variance are then examined with the use of multimode factor analysis which renders an interpretation of the types of products preferred by certain types of buyers in various types of situations.

Results appear to confirm the dominance of interactions over main source effects in contribution to variance. For example, the Situations by Products interaction has been found to be an important determinant of choice behavior for snack and meat products inventories (Belk, 1974a). And this is the effect commonly implied when one refers to situational influences in buyer behavior. However, Lutz and Kakkar (1975) report an interaction between Situations and Products accounting for considerably less variance than found by Belk (1974a). While there are alternative explanations for this discrepancy (e.g., the two studies employed different methods), Lutz and Kakkar's (1975) suggestion that the difference in results may have been due to demand characteristics that were operating in Belk's (1974a) experiment deserves careful investigation to assure future progress along this promising line of research.

The concept of demand characteristics has Lewinian-Festalk connotations (Orne, 1970, p. 226), and it concerns socio-psychological aspects of the experiment which may be used by the subjects as cues to penetrate the ambiguousness or deceptiveness of instructions and to discern the experimenter's expectancies regarding their behavior. Demand characteristics, when allowed to operate unnoticed and uncontrolled by the experimenter, can contaminate an otherwise well-designed experiment, and they can pose strong threats to both internal and external validity. For example, Page (1974) and Page and Scheidt (1971) provide evidence that demand characteristics confounded experiments on the classical conditioning of attitudes (e.g., A.W. Staats and C.K. Staats, 1958) and the so-called weapons effect (Berkowitz and LePage, 1967). Consumer researchers as well have recently addressed themselves to the problem of demand characteristics (e.g., Sawyer, 197Sa,b).

The argument that Belk's (1974a) results may be vulnerable to a demand effect interpretation is founded on the modified behavioral differential questionnaires the study employed. In those questionnaires, the situations vary while the product choices remain constant. When a within-subjects design is employed, this may provide the subjects with the cue that they are "supposed to" shift in their buying choices across the situations. Based on Orne's (1962) suggestion that most subjects are motivated to try to confirm what they believe to be the experimental hypothesis, the result may be an artificially inflated Situations by Products interaction effect. However, while many subjects appear to adopt such a "compliant" role (e.g., Levy, 1967; Sawyer, 1975a), other roles with different effects on the dependent variable have been identified. The "negative" subject role occurs when a subject tries to disconfirm a suspected experimental hypothesis (Allen, 1967; Berkowitz, 1971), while the "faithful" subject attempts to continue in a role that emphasizes adherence to instructions, despite suspicion, for the sake of the validity of the experiment (Fillenbaum, 1966; Golding and Lichtenstein, 1970). Thus, the specific effects of demand awareness depend on adopted subject roles. Demand awareness may cause false positive findings ("compliant" subject role), false negative findings ("negative" subject role), or it may have no important performance effects ("faithful" subject role).

TESTING FOR DEMAND ARTIFACTS

These various considerations led to the following experiment in which both demand awareness and subject role set were manipulated in four conditions. In the first condition, subjects completed a product inventory taken from the Belk (1974a) study and they responded to a brief post-experimental questionnaire which was designed to isolate subjects who were aware of the experimenter's expectation that their product preferences would vary across situations (Control Condition). [This is not to suggest that Belk (1974a) specifically tested such a hypothesis. Rather, based on Lutz and Kakkar's (1975) suggestion that demand awareness led to the greater Situations by Products interaction effects found by Belk (1974a), this was construed to be the experimenter's hypothesis.] The subjects in the second condition were informed about the experimenter's expectation before their completion of the questionnaire (Aware Condition). In the third condition, subjects were instructed to confirm what they thought the experimenter's expectation was (Confirm Condition), while in the fourth condition they were asked to disconfirm it (Disconfirm Condition). The subjects in the third and fourth conditions also completed the post-experimental questionnaire administered to subjects in the first condition.

Since the preponderance of evidence suggests a "compliant'' subject role, the variance accounted for by the Situations by Products interaction should be greater in the Aware Condition than in the Control Condition. Similarly, under the assumption that a substantial number of subjects are demand-aware, this interaction should explain more variance in the Confirm Condition than in the Control Condition. Alternatively, in the Disconfirm Condition, the importance of this interaction should be less than in the Control Condition. If these results should not materialize, the Control Condition's outcome would likely be vulnerable to a demand bias interpretation.

METHOD

The subjects, 101 undergraduates from introductory business courses at Iona College, were assigned to four conditions: Control Condition (24 subjects), Confirm Condition (28 subjects), Disconfirm Condition (25 subjects), and Aware Condition (24 subjects). Each subject was administered the Belk (1974a) snack products questionnaire which consists of ten different snack consumption situations. The subjects were asked to respond to a ten-item (snack products) behavioral differential inventory in each situation. The subjects in the Control Condition received the following general instructions:

"On the following pages you will be asked to check how likely it would be that you would choose snacks in a series of circumstances. For example, friends have invited you to a picnic and asked you to bring some snacks along. How likely would it be that you would choose each of the following snacks in these circumstances: potato chips, popcorn, cookies, fresh fruit, sandwiches, pastries, ice cream, cheese, assorted nuts, and crackers. In the questionnaire, there are ten different circumstances and for each one of them you will be asked how likely it would be that you would choose each of these snacks."

The subjects in the Confirm Condition, Disconfirm Condition, and Aware Condition received these general instructions as well as the following condition-specific instructions:

"As you can imagine I have a certain expectation as to how the results will turn out. Needless to say, I can't tell you about my expectation beforehand, that is, the way I think you will fill out the questionnaire; but while you are filling out the questionnaire you may get some idea as to what you think my expectation is. If you should get an idea, fill out the questionnaire according to what you think my expectation is. If not, fill it out anyway." (Confirm Condition. )

"As you can imagine I have a certain expectation as to how the results will turn out. Needless to say, I can't tell you about the expectation beforehand, that is, the way I think you will fill out the questionnaire; but while you are filling out the questionnaire you may get some idea as to what you think my expectation is. If you should get an idea, fill out the questionnaire opposite to what you think my expectation is. If not, fill out the questionnaire anyway." (Disconfirm Condition.)

"As you can imagine I have a certain expectation as to how the results will turn out. I may as well tell you about my expectation beforehand. I expect that the likelihood of your choosing a particular snack is dependent upon the circumstances. For example, when friends have invited you to a picnic, you may be more likely to choose cheese as a snack to bring along than ice cream. Under different circumstances, however, you may be more likely to choose ice cream as a snack than cheese." (Aware Condition.)

After the subjects in the Control Condition, Confirm Condition, and Disconfirm Condition finished the snack products instrument, they were instructed to complete the following questionnaire:

1. When I presented you with the questionnaire, suppose that I expected that the likelihood of your choosing a particular snack is dependent on the circumstances. While you were filling out the questionnaire, did you expect that this was my approximate hypothesis?

Yes_________ No__________

2.If yes, how certain were you of this at the time you were filling out the questionnaire?

Guessing 1___:2___:3___:4___:5___:6___:7___ Certain

3. As you can see, one thing I am studying concerns your suspicions about the questionnaire. Hake any other comments here about your thoughts, reactions, and suspicions about the questionnaire.

A subject was considered demand-aware if he responded affirmatively to Question 1 and received a score of at least five for Question 2. A score of at least five should be indicative of an acceptable level of certainty in the subject's response to Question 1.

RESULTS

Based on the data secured by the post-experimental questionnaire and the previously stated criteria, 17 out of 23 subjects (73%) in the Control Condition, 10 of 25 subjects (40%) in the Confirm Condition, and 12 of 25 subjects (48%) in the Disconfirm Condition were classified as demand-aware (X² = 5.986; df = 2; p > .05). [Four incomplete questionnaires (one, three, and one for the Control Condition, Confirm Condition, and Aware Condition, respectively) were discarded. Responses to the third item of the post-experimental questionnaire such as "I figured that you wanted to know what snack I would pick under a certain set of circumstances since the types of snack remained the same and only the circumstances changed" were not uncommon.] This suggests that demand-awareness may contaminate results obtained by the conventional assessment of situational effects on buyer behavior. Consequently, further analysis was performed to determine the specific effects of demand-awareness.

For each condition separately, the subjects' choice data were analyzed with the intent to construct estimates of the relative contribution of sources -- as derived from expected mean square formulas specified by a three-way random effects model (Gleser, Cronbach, and Rajaratnam, 1965) -- to variance in the dependent measure. [Since the situations and products in Belk's (1974a) inventory are not random samples of all possible situations and products, a mixed-effects model may appear to be more appropriate. However, in contrast to the random-effects model, with the mixed-effects model one can solve for all the components of variance only if one assumes the triple interaction to be zero. This triple interaction should be meaningful, and consequently it was believed to be less hazardous to assume a random-effects model. At any rate, as Endler and Hunt (1966) point out, the components of variance are comparable for random-effects and mixed-effects models. Thus, even if a mixed-effects model would have been assumed, the interpretation of the results across the conditions would have remained unchanged.]

The results in Table 1 show that the relative contributions of individual sources to variance in product preferences varied markedly across the four conditions. Generally speaking, this indicates that different role sets and levels of demand awareness may contaminate overall results. With respect to the source effect of primary interest, namely the Situations by Products interaction effect, its relative importance differed across the conditions. More specifically, the Situations by Products interaction contributed 1.49% (Disconfirm Condition), 11.49% (Aware Condition), 14.20% (Confirm Condition), and 16.34% (Control Condition) to the variance in the dependent measure. These findings imply that the Control Condition's outcome is very likely not vulnerable to a "negative" subject interpretation but more vulnerable to a "compliant" one.

TABLE 1

RELATIVE CONTRIBUTIONS OF SOURCES TO VARIANCE IN PRODUCT PREFERENCES

DISCUSSION

The considerable proportions of subjects identified here as demand-aware leave previous results (e.g., Belk, 1974a) open to a demand bias interpretation. One could argue that post-experimental questionnaires --especially when lengthy and detailed -- have the undesirable effect of "suggesting" awareness to subjects. However, in light of the very brief questionnaire that was employed in this study, such a possibility appears to be weak. Moreover, as the discussion that follows indicates, the negligible Situations by Products interaction effect obtained in the Disconfirm Condition does not support the notion that the post-experimental questionnaire "suggested" awareness to many of those subjects who were classified as demand-aware.

As far as the specific effects of demand awareness are concerned, the substantially lower variance accounted for by the Situations by Products interaction in the Disconfirm Condition strongly suggests that the Control Condition's result was not suppressed by a dominance of subjects who adopted a "negative" role. Furthermore, this small interaction effect indicates that the subjects in the Disconfirm Condition classified as demand-aware apparently became so soon after they had started to complete the questionnaire and could readily manipulate their answers in accordance with their perception of the experimenter's expectation. There is no compelling reason to believe that the demand-aware subjects in the Confirm Condition differed substantially from the demand-aware subjects in the Disconfirm Condition in regard to overall data manipulation ability and the experimenter expectation recognition pattern. Consequently, if a demand bias interpretation due to "compliant'' subjects is to be ruled out, the variance accounted for by the Situations by Products interaction should have been much greater in the Confirm Condition than in the Control Condition. Quite to the contrary, however, this interaction accounted for slightly more variance in the Control Condition than in the Confirm Condition. This result is plausible in light of the greater proportion of demand-aware subjects in the former condition and the findings which suggest that more subjects adopt a "compliant" role (e.g., Sawyer, 1975a), but it is not necessarily consistent with the outcome of the Aware Condition. In the Aware Condition, the variance accounted for by the Situations by Products interaction was less than in the Control Condition. In accord with Brehm's (1966) reactance theory, some subjects might regard the explicit introduction of the experimenter's expectation as an unwelcome pressure on them. In resisting this pressure, they may then show relatively little of the behavior the experimenter wants.

In sum, this research shows that the conventional assessment of situational effects in buyer behavior may introduce demand characteristics with confounding effects. As indicated by the great proportions of subjects who were classified as demand-aware, the within-subjects design -- where a number of experimental treatments (i.e., situations) is administered to the same subject -- commonly employed in this assessment is particularly demand-prone. This calls for a greater reliance on between-subjects designs in future research. Alternatively, the number of treatments to which an individual subject is exposed should be reduced. The greater the number of treatments in a with-in-subjects design, the more likely is a subject to successfully estimate the experimenter's expectations. While the results render very little evidence in favor of a "negative" subject role and more overall evidence in support of a "compliant" subject role interpretation, this study does not provide conclusive evidence as to the specific effects of operating demand characteristics in the conventional assessment of situational variables. Regardless of specific effects, the researcher's best assumption is probably that data from suspicious subjects are untrustworthy.

REFERENCES

Allen, V.L. "The Effect of Knowledge of Deception on Conformity," Journal of Social Psychology, 69 (1966), 101-106.

Belk, R.W. "An Exploratory Assessment of Situational Effects in Buyer Behavior," Journal of Marketing Research, 11 (May 1974), 156-163(a).

Belk, R.W. "Application and Analysis of the Behavioral Differential Inventory for Assessing Situational Effects in Buyer Behavior," in S. Ward and P. Wright, Advances in Consumer Research: Volume I. Urbana, Ill.: Association for Consumer Research, 1974, 360-370(b).

Berkowitz, L. and A. LePage. "Weapons as Aggression-Eliciting Stimuli," Journal of Personality and Social Psychology, 7 (1967), 202-207.

Berkowitz, L. "The Weapons Effect, Demand Characteristics, and the Myth of the Compliant Subject," Journal of Personality and Social Psychology, 20 (1971), 332-338.

Brehm, J.A. A Theory of Psychological Reactance. New York: Academic Press, 1966.

Endler, N.S. and J.M. Hunt. "Sources of Behavioral Variance as Measured by the S-R Inventory of Anxiousness," Psychological Bulletin, 65 (June 1966), 336-346.

Fillenbaum, S. "Prior Deception and Subsequent Experimental Performance: The Faithful Subject," Journal of Personality and Social Psychology, 4 (1966), 532-537.

Glesner, G.C., L.J. Cronbach, and N. Rajaratnam. "Generalizability of Scores Influenced by Multiple Sources of Variance," Psychometrika, 30 (December 1965), 395-418.

Golding, S.L. and E. Lichtenstein. "Confession of Awareness and Prior Knowledge of Deception as a Function of Interview Set and Approval of Motivation," Journal of Personality and Social Psychology, 14 (1970), 213-223.

Levy, L. "Awareness, Learning and the Beneficient Subject as Expert Witness," Journal of Personality and Social Psychology, 6 (1967), 363-370.

Lutz, J. R. and P. Kakkar. "The Psychological Situation as a Determinant of Consumer Behavior," in M.J. Schlinger, Advances in Consumer Research: Volume II. Chicago, Ill.: Association for Consumer Research, 1975, 439-454.

Orne, M. "On the Social Psychology of the Psychological Experiment: With Specific Reference to Demand Characteristics," American Psychologist, 17 (November 1962), 776-783.

Orne, M. "Hypnosis, Motivation, and the Ecological Validity of the Psychological Experiment," in W.J. Arnold and M.M. Page, Nebraska Symposium on Motivation. Lincoln, Nebraska: University of Nebraska Press, 1970.

Page, M. M. and R.J. Scheidt. "The Elusive Weapons Effect: Demand Awareness, Evaluation Apprehension, and Slightly Sophisticated Subjects," Journal of Personality and Social Psychology, 30 (1974), 468-476.

Sawyer, A. G. "Detecting Demand Characteristics in Laboratory Experiments: The Case of Repetition-Affect Research," in M.J. Schlinger, Advances in Consumer Research: Volume II. Chicago, Ill.,: Association for Consumer Research, 1975, 713-724(a).

Sawyer, A. G. "Demand Artifacts in Consumer Research," Journal of Consumer Research, 1 (March 1975), 20-30(b).

Staats, A. W. and C. K. Staats. "Attitudes Established by Classical Conditioning," Journal of Abnormal and Social Psychology, 57 (1958), 37-40.

----------------------------------------