Consumer Learning Through Experience: a Study and Experimental Paradigm

ABSTRACT - In this study respondents were asked to make a series of choices on a set of stimuli defined by abstract cues. After each choice a reward/penalty was given depending on the quality of the item chosen. Generally, respondents did a poor job of correctly inferring the additive values of the cues from 32 trials. An examination of the patterns of successive choices indicates that respondents were not really attempting to learn but were primarily avoiding negative feedback.

Citation:

Joel Huber and Terry Elrod (1981) ,"Consumer Learning Through Experience: a Study and Experimental Paradigm", in NA - Advances in Consumer Research Volume 08, eds. Kent B. Monroe, Ann Abor, MI : Association for Consumer Research, Pages: 66-70.

Advances in Consumer Research Volume 8, 1981      Pages 66-70

CONSUMER LEARNING THROUGH EXPERIENCE: A STUDY AND EXPERIMENTAL PARADIGM

Joel Huber, Duke University

Terry Elrod, Columbia University

[The authors wish to thank Morris Holbrook and Bob Chestnut for very helpful comments on an earlier draft.]

ABSTRACT -

In this study respondents were asked to make a series of choices on a set of stimuli defined by abstract cues. After each choice a reward/penalty was given depending on the quality of the item chosen. Generally, respondents did a poor job of correctly inferring the additive values of the cues from 32 trials. An examination of the patterns of successive choices indicates that respondents were not really attempting to learn but were primarily avoiding negative feedback.

INTRODUCTION

The question this paper raises is how well people learn to use unfamiliar cues to guide choice when the only source of information is past choices. In a consumer context a distinction has been drawn between experience goods, whose worth can be ascertained only through consumption, and inspection goods, whose worth can be ascertained prior to purchase (Nelson 1970).

Suppose a consumer desires to discover the most preferred item from a set of unfamiliar experience goods. A rather laborious way to find the optimal product is to try each. Such a process would be more difficult still if feedback from each item had random error since then each brand would have to be sampled a number of times to arrive at a stable estimate of its value. If, however, there is a set of cues that are consistently related to the evaluations of the products, the task of finding the best is simplified. The consumer can sample certain items to determine the worths of the cues and then use the cues to find the most preferred item.

This paper presents a paradigm that allows an exploration of people's use of this latter strategy. It estimates the extent to which they can quickly infer the worths of unfamiliar cues from choice experiences and use these inferences to arrive at better choices. The use of the term "cues" is used rather than "attributes" to emphasize their unfamiliarity and hence the lack of priors as to their worths.

An understanding of how consumers assign worths to cues through experience is important because (I) virtually all cues are of unknown significance to a person at some point in time, and (2) the efficiency of a market economy depends upon the consumer's ability to learn preference through purchasing. Consumers who can't learn to utilize cues efficiently to guide choice need more:

a) protection by legislation from deceptive or irrelevant advertising claims;

b) assistance with brand evaluation by independent testing agencies, and

c) training in normative methods of comparison that take into account the consumer's limited ability to evaluate brands.

PRIOR METHODOLOGY - MULTIPLE CUE PROBABILITY LEARNING

The research tradition that has addressed the question of how well people can learn to use unfamiliar cues to guide choice most directly has been called Multiple Cue Probability Learning (MCPL). Relevant reviews by Slovic, Fischhoff and Lichtenstein (1977) and Schmitt, Coyle and King (1976) provide good overviews of what appears to be a relatively mature area of inquiry. This tradition has (1) controlled stimulus outcome and the relation of cues to choice outcome, (2) measured the subjects' prior beliefs about the cues' relations to choice outcome, (3) studied how rate of learning of cue worth is affected by such factors as the number of different cues, the strength and complexity of their relationship to choice outcome (e.g. nonlinearities, interactions), and the value of different types of additional feedback. A common research paradigm has been to control prior beliefs by presenting abstract stimuli to subjects in an experimental setting. The subject is shown a stimulus-object, (e.g. consisting of three bar graphs of certain heights), then the subject is asked to guess the worth of the stimulus-object, and finally the subject is told its worth (often with some stochastic error). This stimulus/ guess/feedback process is repeated 100-400 times. The measure of learning is typically the degree of correspondence between guessed and actual stimulus worths.

While the above paradigm has led to a rich stream of results, it differs in two critical respects from the consumer learning about experience goods referred to earlier. First, in the multiple cue paradigm, notice the subject has no control over the selection of the stimulus object, as s/he might in choosing a soft drink or a detergent. The subject is simply shown the object and asked to assess its worth. Second, in the experimental paradigm the subject is concerned with the accuracy of the judgments rather than the actual worth of the stimulus objects. By contrast, in a consumer trial the worth of the object chosen is important and learning or accuracy may have secondary importance. Thus in consumer choice there is a tension between short term reward and long term information. This tension simply cannot be captured by the traditional multiple cue probability paradigm because choice is passive and the outcome (as opposed to accuracy) is of no material concern to the subject.

A Modified Paradigm

A desire to bring the task characteristics more in line with the consumer context, while still retaining control over cues and outcomes, lead to the development of the following task: (1) allow the subject to choose any one stimulus-object from a set of available ones; (2) inform the subject of the material consequence of the choice made (money won/lost), (3) have the subject repeat this choice/result process 32 times.

Two sets of data were collected. First, the choice/feedback process was administered and monitored by computer, which allowed automatic recording of the subjects' choices, the results of the choices, and the subjects' deliberation times. Second, the subjects completed questionnaires after finishing the learning task. A first set of questions simply asked respondents to evaluate the 16 possible stimuli on a 7-point quality scale that mirrors the feedback given. A second set of questions asked respondents to directly evaluate each level of the four dichotomous cues.

The 16 stimulus objects are shown in Figure 1, and represent all possible combinations of the four two-level cues: line/ no line, large/small, shaded/unshaded, and diamond/square. The name given each stimulus object has a one-to-one correspondence with its geometric configuration, e.g. large corresponds to the suffix 'er' whereas small corresponds to 'ex.'

FIGURE 1

NAMES AND SYMBOLS USED IN THE STUDY

EXPERIMENTAL MANIPULATIONS

Several factors that have been found in the MCPL literature to affect learning of cue-worths were manipulated.

Relative Importance of the Cues

Respondents in MCPL tasks have been shown to have prior biases with respect to the amount of information in different cues (Dudycha and Naylor 1966, Janke 1972, Castellan 1974). The current experiment included four levels of dispersion of weights among the cues, ranging from equal importance to highly unequal importance (in which one cue alone accounted for 80% of explainable variance in stimulus worth.

Absolute Worth of Cues

Stated worths of chosen objects contained two error components. First, repetitive selection of the same stimulus yielded different choice outcomes (as in a consumer context, where repeat purchases of a brand do result in different degrees of satisfaction). This source of error is referred to here as 'stimulus error,' and has been found in MCPL tasks to inhibit learning of worths of both stimuli and cues. Second, the cues did not perfectly predict even the expected worth of a stimulus object. This source of error, 'systematic error,' was thought to inhibit the learning of cue, but not object, worth. Each of these sources of error was manipulated as a two-level factor. These manipulations taken together created a 2 (stimulus error) by 2 (systematic error) by 4 (relative cue weights) full factorial design of 16 cells.

The task was performed by 48 MBA students at a major university and typically took 15-20 minutes to complete. These business students were challenged by the task, responding to the experiment as a test of pertinent business-related skills. The 48 respondents allowed three replications per cell. The assignment of weights to cues was rotated across these three replications to give each cue a roughly equal chance, across subjects, of having a given weight.

RESULTS

The main result is that the respondents did not learn the worths of either the stimuli or their cues very well. This result was independent of the experimental manipulations and is based upon analysis of both the questionnaires and the choice process.

Poor Learning as Evidenced by Questionnaires

The measures of learning, as are those in much of the MCPL literature, are based on Brunswik's (1956) lens model as formalized by Dudycha and Naylor (1966). These measures reflect correlations between several estimates of the stimulus worths: (1) Direct estimates were supplied by the questionnaire, (2) Bootstrapped estimates were the predicted worths of the stimuli provided by regressing the direct estimates against the cues, (3) Compositional estimates were derived by combining the ratings of each cue. The implied worth of an object-stimulus was then the sum of the ratings of its cue-levels. Since ratings of the geometric shapes were obtained separately from the ratings of the letters, two separate compositional indices were created.

Given these four subject-based estimates of stimulus-object worths, what measure should be used as a criterion to evaluate the degree of learning reflected in these measures? One could regress each of these measures against the true worths of the stimulus-objects but these true worths are not perfectly related to the cues due to the presence of random error. We therefore calculated a 'fairer' criterion estimate of the stimulus worths by computing those cue weights that would be inferred by a perfect OLS processor given the sequence of chosen stimuli and feedback.

These optimal retinas of the stimuli were obtained by regressing the feedback levels against the cue levels scored as dummy variables. The resulting predicted stimulus worths were optimal in the sense of being the best predictors of the particular stimuli chosen. That is, if a respondent concentrated choices among five or six stimuli, the optimal values reflected the fit of the additive model from among these choices. These scores might be quite different from those generated from a broad selection of stimuli. Thus the optimal scores are relative to the particular choices and to a certain extent control for whether a respondent used a high-learning strategy with many different items chosen or a low-learning strategy. It should also be noted that the optimal values can, in theory, be perfectly reproduced by the subject in that each respondent had the information needed to produce a scale that correlates perfectly with the optimal values.

TABLE 1

AVERAGE CORRELATION OF SUBJECT-BASED ESTIMATES OF STIMULUS WORTHS WITH "OPTIMAL ESTIMATES"

TABLE 2

AVERAGE CORRELATIONS AMONG SUBJECT-BASED ESTIMATES OF STIMULUS WORTHS

The four subject-based estimates of stimulus worth: direct, bootstrapped, shapes and letters, were correlated with the optimal scales for each individual. The means and standard deviations across the 48 respondents are shown in Table 1. It is noted that, except for the direct estimates, which are predictably unreliable, the three scales correlate about 0.3 with the optimal model. This translates to an R² of less than 10%.

The first question raised by this relatively low performance is, how it can be so much lower than typical correlations found in either compositional or decompositional models? The answer to this question can be found by examining the correspondences between the various subjective scales shown in Table 2. These more respectable correlations indicate that consumers are internally consistent. But this internal consistency may greatly overstate actual association with the true worths of the stimuli. Thus it appears that respondents were consistent with their own ratings but these had relatively little correspondence with reality. In the next section we explore the pattern of actual choices made in an attempt to discover the strategies used and why they were unsuccessful in uncovering the true values.

LACK OF LEARNING IN CHOICE PATTERNS

Some of the reasons for the poor learning become apparent as one examines the patterns of choices through the 32 choice-feedback trials. While no clear criterion of learning is available in this case, one would expect that choices might reflect a partly systematic search for the best stimulus object, and that subsequent choices would be somewhat consistent with information obtained from prior choices.

Lack of Search for Structure

All the stimuli differ from each other on between one and four cues. By choosing a stimulus that differs from the previous one on only one cue, the change in feedback is an estimate of that cue's worth. We term such switching on only one cue on two consecutive choices an experimental test. The value of such a strategy when there is no error in feedback is clear: by ignoring the cues, all 16 brands must be tried before the best brand is identified, but by correctly inferring an additive relationship between cues and stimulus worth and 'experimentally testing' the cues, only 5 choices are needed to determine the best brand. When feedback contains error, the best brand can never be identified with certainty by either strategy, but even if the subject decides to try all brands in search of the best, trying them in an 'experimental testing' order would yield valuable information. Did respondents in fact employ such a strategy? The answer is no, as is shown by Table 3, which compares the number of cues switched between consecutively chosen stimuli if all stimuli were chosen randomly (and with equal probability) with the frequency observed with our respondents. Note that the percentage of single-switch choices (experimental tests) is virtually identical to that expected under the chance model. This percentage (25%) was unexpectedly small given the high level of information that can be derived from just changing one cue. Instead, most of the switches (30%) involved changing two cues, a strategy that has limited value from an informational standpoint and is only marginally efficient as a way to find a brand that is quite different. High-aversion strategies occurred relatively rarely, perhaps because of the search difficulty in finding the item that differs over three or four of the cues. Not surprisingly, perhaps, zero switches ('brand loyalty') occurred much more frequently than expected by chance. Overall, there is little evidence that respondents acted as "intuitive scientists" by searching for structure using controlled experiments. Furthermore, as the next section shows, even when "experiments" were used their results were ignored.

TABLE 3

RELATIVE FREQUENCY OF NUMBER OF CUES SWITCHED

Lack of Response to Structure

Figure 2 shows that, when subjects did switch on only one cue, their next choice often conflicted with the information obtained on that cue. If switching on one cue resulted in a positive improvement in choice outcome, then one might expect the subject to retain that level on the subsequent choice. Subjects in fact did this 65%-78% of the time, depending upon the magnitude of the change in outcome. If switching a cue resulted in a negative change in choice outcome, then one would expect subjects to switch back to the original one on their subsequent choice; they in fact did so less than two-thirds of the time.

FIGURE 2

REACTIONS TO POSITIVE AND NEGATIVE CHANGES IN FEEDBACK WHEN EXACTLY ONE CUE IS CHANGED (EXPERIMENTAL TEST)

Evidence of Sensitivity to Immediate Feedback

What, if anything, did influence the number of cues switched on consecutive choices? There are three likely candidates. First might be the outcome on the last item chosen--the higher its value, the greater the probability that the next choice will be highly similar. Another possibility is the change in outcome for the last two choices. One would expect smaller change following an increase in choice outcome than following a decrease. Finally, one might expect experience to result in switching on fewer cues; that is, the respondent might, with experience, eliminate some stimuli from consideration.

In the present study, current feedback accounted for virtually all of the accountable variance in the number of cues switched. Thus, once current feedback is accounted for, the change in feedback was not significant and task experience became much less so. The effect of current feedback is shown in Figure 3.

SUMMARY AND DIRECTIONS FOR FUTURE RESEARCH

The foregoing has described an experimental paradigm that modifies the multiple cue probability learning task to better approximate that aspect of consumer learning that occurs through the experience of the product. Different ways were tried to get respondents to give the values of cues that correspond to the feedback received on the 32 purchase/ feedback occasions. While these different measures had reasonable correlations among themselves (internal validity), none accounted for more than 10% of the variance (external validity) that could have been realized had the respondents been OLS processors. The patterns of choices provided some explanation for the lack of apparent learning in the task. Respondents did not use what would have been the moat efficient way to measure cues worths, experimental tests, any more than would be expected with random selection of stimuli. Further, where experimental tests did occur, subsequent choice was in accordance with the feedback from the test in only 2/3 of the cases. Indeed, the best predictor of the number of cues switched was simply the value of the last time chosen--the more negative its feedback, the more dissimilar the next item chosen. Perhaps as little learning occurred because respondents were not trying to learn but simply avoiding bad products, a kind of aversive random walk.

FIGURE 3

EFFECT OF CURRENT FEEDBACK ON THE NUMBER OF CUES SWITCHED ON NEXT CHOICE

While the results from this exploratory study are too tentative to be applied directly to consumer learning in the marketplace, the importance of learning in that context is great enough to warrant research into the following questions:

Can Subjects Be Induced to Learn Cue Worths?

Several factors would likely serve to increase the level of cue learning. First, the number of stimuli can be increased relative to the number of cues, thus forcing attention to cues rather than global Judgments on stimuli. Second, the consequences of choice can be increased. This change would increase the cost of a 'try all brands and see' strategy. Finally, more cue learning might occur with messages (e.g. ads) to orient respondents to the different cues.

Does Prior Knowledge Affect Learning?

Abstract choice objects were employed in this study, as in the MCPL literature, to minimize the effects of prior learning and, most importantly, to control choice outcomes. An alternative might be to use hypothetical brands of a real product class (as in conjoint analysis), measure (rather than control) the subjects' priors concerning cue worths, and then study the change in these priors as a result of feedback on choices. Alternatively, one might use brand names and measure prior beliefs concerning both cue (attribute) and brand worths.

Thus, the current study, while tentative and largely exploratory, suggests that consumers may have difficulty in correctly inferring the values of purchase cues from product experience. It is hoped that future research will shed more light on this issue of concern to regulators, practitioners and consumer researchers.

REFERENCES

Brunswik, E. (1956), Perception and the Representative Design of Experiments, Berkeley: University of California Press.

Castellan, N. John (1974), "The Effect of Different Types of Feedback on Multiple-Cue Probability Learning, Organizational Behavior and Human Performance, 11, 44-64.

Dudycha, L.W. and Naylor, J.C. (1966), "Characteristics of the Human Inference Process in Complex Choice Behavior Situations,'' Organization Behavior and Human Performance, 1, 110-128.

Janke, Mary (1972), "Effect of Stimulus and Instructional Variables in Ambiguous Concept-Attainment Task," Journal of Experimental Psychology, 93, 21-29.

Nelson, P. (1970), "Information and Consumer Behavior," Journal of Political Economy, 78, &0-53.

Schmitt, Neal, Coyle, B.W. and King, L. (1976), "Feedback and Task Predictability as Determinants of Performance in Multiple Cue Probability Learning Tasks," Organizational Behavior and Human Performance, 16, 338-402.

Slovic, Paul; Fischhoff, B., and Lichtenstein, S. (1977), "Behavioral Decision Theory." Annual Review of Psychology.

----------------------------------------