Testing of Buyer Behavior Models

Citation:

Donald R. Lehmann, John U. Farley, and John A. Howard (1971) ,"Testing of Buyer Behavior Models", in SV - Proceedings of the Second Annual Conference of the Association for Consumer Research, eds. David M. Gardner, College Park, MD : Association for Consumer Research, Pages: 232-242.

Proceedings of the Second Annual Conference of the Association for Consumer Research, 1971     Pages 232-242

TESTING OF BUYER BEHAVIOR MODELS

Donald R. Lehmann, Graduate School of Business, Columbia University

John U. Farley, Graduate School of Business, Columbia University

John A. Howard, Graduate School of Business, Columbia University

[Part of the support for this research was provided by the Columbia University Graduate School of Business Research Fund.]

[John A. Howard and John U. Farley are professors and Donald R. Lehmann is an assistant professor at Columbia University Graduate School of Business]

Comprehensive models of buyer behavior, such as those proposed by Howard-Sheth and Nicosia, have recently received considerable attention in the marketing literature [1,3]. These models are basically attempts to provide a framework (usually portrayed as a flowchart) which explains the interactions of the psychological, socio-economic, and situational factors at work in consumer decision-making processes. While these models have substantial conceptual appeal, serious problems exist in actually testing them. This paper will attempt to discuss the testing of such models first by summarizing two tests of the Howard-Sheth model, and then by discussing some general problems of buyer behavior model validation and refinement

TEST #1

The first major test of the Howard-Sheth model concerned a branded grocery product [53. Faced with the problem of testing the model in its entirety, the authors chose to use an econometric approach. Thus variables which both influence and are influenced by other variables are labeled endogenous, while variables which influence but are not influenced by other variables are labeled exogenous. The model can thus be seen as a system of simultaneous equations, with each endogenous variable serving as the dependent variable in one of the equations.

Using this viewpoint, the data from a sample of 693 housewives was analyzed. Both ordinary least squares and two stage least squares regression analysis were performed. The results, though obviously affected by considerable noise, were generally favorable.

TEST #2

The second full-model test of the Howard-Sheth Model was in many ways a replication of the first [5,103. The data was collected in three waves from a special panel of 200 to monitor the test-market introduction of a new frequently-purchased product in two cities in Argentina. The form of the model used is depicted by the flowchart in Figure 1, and the variable definitions are presented in Appendix 1. The analysis of this data proceeded in two basic ways.

The first major portion of the analysis concerned measuring the potential of the new product. In order to do this, changes in the levels of the key variables were studied over the six-week inter-interview periods. As hoped, Attention increased from O on wave 1 to 63. 2/o of the sample on wave 3, and Purchase reached 15.3% on wave 3. Similarly, the percent of the sample who exhibited "brand impact' (positive Attention, Brand Comprehension, or Attitude) reached 74.4% on wave 3. These results proved sufficiently encouraging to recommend further development of the product.

FIGURE 1

ORIGINAL MODEL

TABLE 1

SUMMARY OF REGRESSION RESULTS (OLS)

The second major portion of the analysis concerned testing of the model itself rather than the product. Proceeding similarly to the first test, the model was structured in terms of ten equations (one in which each of the 10 endogenous variables in Figure 1 was the dependent variable). Analysis consisted of two basic stages.

The first stage of the analysis concerned taking the model as given and examining both ordinary and two-stage least squares crosssectional regressions. As expected, the ** coefficients between endogenous variables were positive. On the other hand, the exogenous variables t effects are less well-defined. This conclusion is supported by reduced form regressions involving purchase. Here using all variables resulted in R2's of about .3, while using only exogenous variables led to R2 t S on the order of .04. In addition, it was clear that the market input variables were more valuable than demographics as predictors.

In terms of overall predictive power, the results are somewhat mixed. The R2's (Table 1) are in general an improvement over the first test, but not the large improvement hoped for. In addition, there is wide disparity in R2's between equations. The equations involving the cognitive variables (Attitude, Brand Comprehension, Satisfaction, and Purchase) were reasonably good (R2's greater than .25). With the exception of Attention, however, the equations involving the informational variables resulted in low R2ts (less than .10). Thus further refinement on the informational side of the model is needed.

The second part of the analysis of the model consisted of an attempt to revise the model in order to improve its predictive performance. To do this, variables which seemed likely to improve prediction of the dependent variables because of their pairwise correlations were used to form a "better fitting" model. The variables entered in each equation were constrained by the causal ordering of the model so that Purchase, for example, was not allowed to be an independent variable in the equation involving Attitude.

As expected, the results from this form of the model are better. For example, the R2 on the Purchase equation increased from .11 to .32 on wave 3 for the sub-sample of brand-aware consumers. Other changes were in general less spectacular. Moreover, the R2's involving the informational variables were still noticeably lower than those involving the cognitive variables. Since certain of the equations had little conceptual appeal (for example, Satisfaction is not assumed to affect any of the other variables), a revised model combining the virtues of both models was constructed. This model differed from the original in two main ways: it was more recursive and it allowed more exogenous variables to be used as independent variables (Figure 2).

The conclusions of this study were twofold. First, the product was deemed to be successful. Second, the model itself was maintained to be substantially valid, although still in need of considerable refinement.

TENTATIVE CONCLUSIONS

Analysis of the two previously discussed studies as well as some other work [2,9,123 leads to several conclusions.

1. The model has substantial appeal, but still needs extensive testing. This testing can probably be best performed by testing various segments of the model separately.

2. An information system designed to monitor a new product could be developed on the basis of a reduced set of six endogenous variables: Attention, Brand Comprehension, Attitude, Intention, Purchase, and Satisfaction. Exogenous variables must be determined by experience with the product category in question.

FIGURE 2

REVISED MODEL

3. Buyer behavior models are more recursive than their present flowchart versions imply. In other words, variables which occur early in the system (Attention, for example) are related to variables that occur late in the system (such as Intention) as well as to intervening variables (such as Brand Comprehension). The intervening variables can thus be considered to be imperfect filters of the information contained in variables earlier in the system.

4. The noise level in the data is a large problem in estimating relationships. Respondents to surveys designed to gather the massive data needed to test the model give noisy data through a combination of low initial commitment and fatigue. Thus fine differences in underlying structures will be extremely difficult to detect.

5. The distinction between endogenous and exogenous variables is not sharp. For example, "advertising exposed to" is assumed to be exogenous. Yet since it is usually measured by direct questioning of the respondent, it is likely to be affected by Attention, Attitude, and other endogenous variables, and thus is itself endogenous. On the other hand, Stimulus Ambiguity could well be considered to be an exogenous variable.

6. Better operational definitions of the variables are needed. Clearly endogenous variables such as Perceptual Bias are difficult to define. Yet even such a "hard" variable as Purchase can be defined many ways: "last brand bought", "brand most frequently purchased", "percent of purchases allocated to each brand", etc. Until operational definitions are agreed upon, it will be very difficult to apply results of one study to another situation.

7. The direction of causation is not clear. The question of which comes first -- a change in behavior or a change in attitude -- can be asked of all relationships postulated by the model.

8. Purely mechanical applications of the model at this stage in its development will not be rewarding. Most of the information gained from testing the model was obtained by "tinkering" with the data and the model rather than from simply running the appropriate regressions. The value of this tinkering seems likely to be substantial for some time to come.

TESTING TO BE DONE

A tremendous amount of testing of the Howard-Sheth model remains to be done. At least four areas of investigation seem especially relevant

1. Trying different operational definitions. For example, it remains to be seen whether objective but limited brand comprehension measures are more appropriate than subjective evaluations provided by the respondent himself.

2. Testing non-linear forms of the model. Largely for convenience, previous tests of the model have assumed that links between the variables are linear. Yet logically there is no reason to assume linear relationships. The poorly explained informational variables seem to be especially good candidates for trying out non-linear relationships.

3. Revising the model so that it is exactly identified. This would allow the use of the B's to deduce relative strength of influence. However, unless the model in theory is exactly identified, arbitrarily making it so is highly questionable.

4. Trying lagged forms of the analysis. To date, most of the work on the Howard-Sheth model has been centered on cross-sectional analysis.

Specifying or deducing a lagged structure would have the advantage of making the causal priority clear. Unfortunately, the lag between many of the variables may be only seconds or even fractions of seconds, and hence impossible to deduce in data gathered every 4-6 weeks.

DETERMINING THE BEST MODEL

As testing proceeds, competing models will appear and the question of which is best will naturally arise. Unfortunately the determination of the best model is extremely difficult. The reason for this difficulty is that there is no single criterion for deciding which is the best model, or even whether a particular model is good or not good. While many criteria exist, they can be grouped into two main categories: subjective and objective. There are 4 widely used subjective criteria:

1. Common Sense (Introspection)

This criterion basically suggests that unless a model has conceptual appeal, it is not good. While easy and inexpensive to apply, this criterion has some obvious shortcomings. The most obvious of these is that common sense is a very ambiguous criterion. It is also a criterion which seems likely to exclude any "new" knowledge: ("The world cannot be round", "sales cannot be positively related to price", etc.)

2. Agreement with Known Truths (Literature Search)

This criterion, which consists of looking for the results of past studies, appears to be both scholarly and tedious. Like the common sense criterion, however, this criterion is neither unambiguous (because of the numerous conflicting results in the literature) nor amenable to accepting new knowledge.

3. Simplicity

This criterion argues that the less complicated a model is, the better. Unfortunately, what seems simple to one judge may seem complicated to another.

4. Informational Value

This criterion maintains that unless a model has informational value, it is not "good". Obviously a model which cannot be interpreted is of little value. Here again, however, the criterion is ambiguous.

In addition to subjective criteria, there are at least three objective criteria for testing a model:

1. Goodness of Fit

One criterion which appears to be unambiguous is to select the model which has the highest goodness of fit measure, such as an R2 in regression analysis. This criterion is obviously most applicable when the goal is prediction (forecasting) rather than explanation. Unfortunately, goodness of fit measures are subject to both random influences and innumerable statistical biases. Moreover, if a model consists of multiple equations, it is not clear how to combine the individual goodness of fit measures to form an overall goodness of fit measure. (For example, is a two-equation model with R2's of .5 and .4 better, equal to, or worse than a model with R2's of .8 and .1?)

2. Structural Parameters

A very important criterion for testing a model concerns the structural parameters, such as the $'s in regression analysis. This criterion is especially important when the goal is exploration of a certain situation rather than prediction. The structural parameters should be significant, and in addition, the size and signs of these parameters should be "reasonable". Unfortunately it is not clear exactly how to measure the average significance and reasonableness of a set of parameters.

3. Predictive Validity

A very useful criterion for testing a model is to see whether it is predictively valid. This criterion states that conclusions (hopefully surprising) deduced from a model should be tested, and if the conclusions are false, the model is to he rejected. Unfortunately there are often an infinite number of potential conclusions to test. Furthermore, satisfying this criterion is a necessary but not sufficient condition for accepting a model.

Thus it is obvious that complete testing of a model involves multiple criteria. Moreover, it seems unlikely that one model can dominate another on all the criteria. (For example, a simpler model almost by necessity must have less informational value.) Hence choice of a "good" model depends heavily on the relative importance a judge attributes to these or other criteria.

CONCLUSION

The first two extensive tests of the Howard-Sheth model lead to two very different conclusions. First, the model seems to have substantial validity. Second, the model is still in need of considerable refinement and subsequent testing. It seems obvious that this refinement and testing will be both frustrating and worthwhile. The end result seems likely to be a model with a wide range of applicability.

APPENDIX 1

ARGENTINA STUDY VARIABLE DEFINITIONS

REFERENCES

Aaker, David A., "Using Buyer Behavior Models to Improve Marketing Decisions," Journal of Marketing, 34 (July 1970) 52-7.

Day, George L., "Buyer Attitudes and Brand Choice Behavior," Ph.D. dissertation, Graduate School of Business, Columbia University, 1968.

Ehrenberg, A.S.C., "Toward an Integrated Theory of Consumer Behavior," Journal of the Market Research Society, 21 (October 1969), 305-37.

Farley, John U., John A. Howard, and Donald R. Lehmann, "Test Markets and Buyer Behavior,'t working paper, Graduate School of Business, Columbia University, 1970.

Farley, John U., John A. Howard, and Donald R. Lehmann, "After Test Marketing, What?" 1970 Proceedings of the Business and Economic Statistics Section, American Statistical Association (Annual Meeting, Detroit, December 27-30, 1970), Washington, pp. 288-96.

Farley, John U. and Winston L. Ring, "An Empirical Test of the Howard-Sheth Model of Buyer Behavior," Journal of Marketing Research, 7 (November 1970), 427-38.

Howard, John A., "Structure of Buyer Behavior," working paper, Graduate School of Business, Columbia University, Spring, 1970.

Howard, John A. and Jagdish N. Sheth, The Theory of Buyer Behavior, New York: John Wiley and Sons, 1969.

Lampart, Schlomo, "Word of Mouth Activity During the Introduction of a New Product," Ph.D. dissertation, Graduate School of Business, Columbia University, 1970.

Lehmann, Donald R., Terrance V. O'Brien, John U. Farley, and John A. Howard, "Empirical Contributions to Buyer Behavior Theory," working paper, Graduate School of Business, Columbia University, 1970.

Nicosia, Francesco, Consumer Decision Processes: Marketing and Advertising Implications, Englewood Cliffs, N.J.: Prentice Hall, Inc., 1966.

O'Brien, Terrance V., "Information Sensitivity and the Sequence of Psychologican States in the Brand Choice Process," Ph.D. dissertation, Graduate School of Business, Columbia University, 1970.

----------------------------------------