Fellow's Award Speech Musings on Method in Consumer Research

Citation:

Paul E. Green (1992) ,"Fellow's Award Speech Musings on Method in Consumer Research", in NA - Advances in Consumer Research Volume 19, eds. John F. Sherry, Jr. and Brian Sternthal, Provo, UT : Association for Consumer Research, Pages: 9-11.

Advances in Consumer Research Volume 19, 1992      Pages 9-11

FELLOW'S AWARD SPEECH

MUSINGS ON METHOD IN CONSUMER RESEARCH

Paul E. Green, University of Pennsylvania

INTRODUCTION

I'm deeply grateful to have been elected a fellow of the ACR. As an unabashed and sometimes outspoken methodologist, I was rather surprised to see my name added to a distinguished list of bonafide consumer researchers. As a tottering sexagenarian, I was particularly happy to join the ranks while I could still climb the stairs to the podium.

Even in an age of naturalism and postmodernism (Lutz 1989), traditional methodology still holds a place in consumer research. Indeed, alongside the venerable ANOVA and ANCOVA models, we are becoming increasingly familiar with the path diagrams and related tools of covariance structure analysis.

One early thought that I had was to regale you with a long list of technical advances in methodology~a kind of showcase of what may be in store for the nineties. (A 30-minute cap on the length of the talk fortunately quashed that notion.)

So, quite the contrary, my topic is on the role of simple models in consumer research. Over the past several years, I've become impressed with how good simple, robust models of decision making can be. But before getting into that subject, let me digress a little and talk about robustness in other contexts.

ROBUST QUALITY DESIGN

Today's quality control engineers speak almost as one voice in extolling the virtues of robust quality design, particularly Taguchi methods (Dehnad 1989). Quality engineers call the uncontrollable variables that cause characteristics of a product to deviate from their target values noise factors. These factors consist of: (a) external variations in the product's operating environment (e.g., temperature, humidity, or vibrations); (b) variations in product parameters during manufacturing; and (c) product deterioration over time and use.

Robust design engineers seek to produce a product that is robust with respect to all three categories of noise. This idea is operationalized by setting up experiments where the problem is to find engineering designs that maximize the signal to noise ratio while still keeping the parameters' mean values on target.

Conjoint analysts will be happy to learn that Taguchi and his followers are avid users of orthogonal arrays when designing the necessary experiments for building in quality robustness.

ADAPTIVE PROCESS CONTROL

My second example is also from the production sector. Since the early fifties, process engineers have been interested in response surface methods for optimizing productivity. Simple mathematical models~typically generalized quadratic functions~are chosen to represent the effect of a key set of control variables on the yield of some chemical or physical process.

Experiments are set up to find settings of the control variables that optimize process yield. However, that's only a start. Processes change over time and generate the need for continuous fine tuning. This is where the methodology of EVolutionary OPeration (or EVOP, for short; Box and Draper 1969) comes in.

EVOP is a management tool in which continuous investigation and experimentation serve as the basic mode for running a production facility. In EVOP, a process should be operated in such a way as not only to produce saleable product but also to produce information on how to improve the product in the future.

EVOP does this by carefully conducting on-line experiments in which process control variables are systematically varied during actual production runs. But they're varied in such a way that usable product is still produced. As environmental variables change, the control variable settings adapt in a more or less continuous approach to optimal response seeking.

The model of an EVOP process is (generally) a simple statistical, low-order polynomial representation that may have little relationship to the underlying chemistry of the process. In the words of the psychologist, Paul Hoffman (1960), it is a paramorphic model of the process. And, it's not unlike our simplified models for representing human judgment and choice. Again, highly fractionated factorial designs are used in implementing the EVOP experiments.

STATISTICAL ESTIMATION

My third example is from the statistics literature. Today's statisticians are becoming increasingly interested in robust methods for computing measures of location and dispersion (Huber 1981), as well as fancier estimation procedures for such multivariate techniques as discriminant analysis, factor analysis, cluster analysis, and multidimensional scaling (Spence and Lewandowsky 1989; Heiser 1987; Gnanadesikan and Kettenring 1972).

In statistics, the robustness of a procedure is defined as the property that makes the technique's answers reasonably insensitive to the kinds of departures from ideal assumptions that often occur in practical situations. When statisticians work with models, they are usually less interested in whether the model is "true" and more interested in whether it is illuminating and useful under a variety of contingencies.

Much of this research has focused on the modification of least squares estimation so that outliers have less influence on final parameter estimates. Order statistics, such as the median or quantile, and various kinds of trimmed means are typically employed. Much of the work on robust statistical methods has gone on, in parallel, with computer intensive methods like flexible regression (Cleveland 1979; Rust 1989) and the bootstrap resampling method.

In sum, the development and application of robust models and methods is an ongoing enterprise in a variety of disciplines.

MULTIATTRIBUTE MODELS OF JUDGMENT AND CHOICE

One thing in which virtually all consumer researchers have interest is multiattribute models for judgment and choice. Over the years, the contributions of Paul Hoffman and colleagues in policy capturing models, the social judgment models of Hammond, and Norman Anderson's functional measurement have served as important underpinnings in today's consumer research.

One stream of research that has emanated from this earlier work has resulted in the development of pragmatic tools, such as conjoint analysis. Another stream of research has focused on non-compensatory models, such as EBA, lexicographic, and conjunctive/disjunctive models. What does model robustness have to do with the topic of multiattribute decision making?

In 1974 a now-classic paper by Dawes and Corrigan appeared. Their topic was as central then as it is now-namely, what is the role of linear models in decision making? For example, the problem context could entail predicting a graduate student's first-year grade point average, given knowledge of her scores on various aptitude tests, undergraduate record exam scores, peer ratings, years of work experience, and so on.

Dawes and Corrigan reported that a large number of empirical studies showed that simple linear, additive models performed well, typically better than the decision maker herself. In general, Dawes and Corrigan found that linear models predicted well when:

1. Part-worths are conditionally monotonic with changes in each attribute's levels. Hence, once a suitable ordering of the attribute's levels is found, the ranking of part-worths with changes in the levels of the attribute does not depend upon the levels that other attributes assume.

2. Errors in the measurement of the attribute levels themselves tend to linearize functions that may be originally highly nonlinear.

3. Relative weights in the model are not affected by increases in the general error term (as is the case in multiple regression).

Later, a fourth condition was added-that the predictor variables are positively correlated, on average (Green and Devita 1975).

Many judgmental and preference tasks approximate at least the first three conditions. If so, Dawes and Corrigan found that the quality of prediction (as measured by a product moment correlation) was little affected by the precision with which the attribute importance weights were measured. Even equal weights worked very well.

MORE RECENT RESEARCH

Much has happened in multiattribute decision making since Dawes and Corrigan's 1974 paper. For example, the part-worth, ANOVA-type model has become popular in conjoint analysis. Even here, however, the main-effects, additive version often cross-validates as well as configural models that contain interaction terms.

Decision analysts have not given up, however, in seeking conditions that would topple the humble main effects, additive compensatory model. For example, in the context of multiattribute choice, as opposed to judgment prediction, the elimination of dominated options typically produces negatively correlated attributes. It is here where predictions would be expected to be more sensitive to the attribute weights.

Experimental evidence by several researchers (e.g., Levin et al. 1983; Green, Helsen, and Shandler 1988; Moore and Holbrook 1989; Elrod, Louviere, and Davey 1989) indicates that models calibrated in orthogonal environments still appear to predict well in more realistic validation settings where dominated options have been removed. The findings of Elrod, Louviere, and Davey are particularly compelling, since their validation sets employed maximally negatively correlated options.

Another facet of the model robustness problem involves the employment of non-compensatory models, such as the lexicographic, conjunctive, EBA, and phased models. These classes of models might be expected to pose a severe challenge to compensatory model approximations. Still, Johnson, Meyer, and Ghose (1989) found that compensatory models with selected interactions did fairly well in mimicking the results of non-compensatory processing, at least in the simulated environments of their experiments.

Overall, orthogonal designs and the relatively simple part-worth models of conjoint analysis appear to do a reasonably good job of preference modeling-at least in the practical world of commerce, where time pressures and less-than-docile respondents are the order of the day. In noisy and frenetic applications, the consumer researcher may simply not have the luxury of using more delicate models and more lengthy data collection techniques.

PLURALISTIC RESEARCH

What is the moral of this little stroll down robust modeling lane? For what they may be worth, here are a few observations:

1. Statisticians are now supplying us with an increasing array of tools for battling noisy data and for isolating outlier culprits. It behooves us to learn some of these methods and apply them to our own research.

2. From the quality control engineers and the process optimization chemists, we should appreciate the value of simple (but robust) models of complex processes. Orthogonal arrays play a major role in both of these experimental design strategies.

3. Closer to home, paramorphic models, like the linear models of Dawes and Corrigan or the part-worth models of the conjoint analyst, appear to provide robust ways for coping with the exigencies of applied consumer judgment and choice problems.

4. But to be able to predict reasonably well is not necessarily to understand the phenomena being predicted. There's plenty of room in consumer research for seeking a deeper comprehension of the phenomena of interest to us.

So, what does this excursion into robust models mean for consumer research generally? It seems to me that consumer researchers will have to get used to both the esoteric and pragmatic side of our natures. There are constituencies out there who want robust models that can make reasonably good predictions. And, there is also a need for more delicate and sensitive models that can explain some of the nuances of decision making and other phenomena of interest to us. Like Rich Lutz earlier stated in his ACR presidential address, let me say that I'm all for research pluralism, too. But, then again, what red-blooded consumer researcher wouldn't be?

Sometimes these two motivations-the theoretical and the pragmatic~will merge and lead to a high-impact result; that is, an idea that is both intellectually exciting and appealing to the practitioner. More often than not, proponents of each approach will go their separate ways with (let us hope) mutual respect, if not endearment.

And what about the methodologist~that back room mole who always seems to be a step or two removed from the action? Not to worry. We'll continue to ply our trade as long as there's another variance to compute, another path diagram to draw, another hypothesis to test.

REFERENCES

Box, George E. P. and Norman R. Draper (1969), Evolutionary Operation. New York: John Wiley & Sons.

Cleveland, William S. (1979), "Robust Locally Weighted Regression and Smoothing Scatterplots," Journal of the American Statistical Association, 74 (December), 829-836.

Dawes, Robyn M. and Bernard Corrigan (1974), "Linear Models in Decision Making," Psychological Bulletin, 81, 95-106.

Dehnad, Khosrow, ed. (1987), Quality Control, Robust Design, and the Taguchi Method. Pacific Grove, CA: Wadsworth.

Elrod, Terry, Jordan J. Louviere, and K. S. Davey (1989), "How Well Can Compensatory Conjoint Models Predict Choice for Efficient Choice Sets?", Working Paper No. 89-18, University of Alberta (November).

Gnanadesikan, R. and John R. Kettenring (1972), "Robust Estimates, Residuals, and Outlier Detection with Multiresponse Data," Biometrics, 28, 81-124.

Green, Paul E. and Michael T. Devita (1975), "The Robustness of Linear Models Under Correlated Attribute Conditions," Proceedings of the American Marketing Association, Rochester, NY (August), 108-111.

Green, Paul E., Kristiaan Helsen, and Bruce Shandler (1988), "Conjoint Internal Validity Under Alternative Profile Presentations," Journal of Consumer Research, 15 (December), 392-397.

Heiser, William J. (1987), "Multidimensional Scaling with Least Absolute Residuals," paper presented at the First Conference of the International Federation of Classification Societies. Aachen, Germany (July).

Hoffman, Paul J. (1960), "The Paramorphic Representation of Clinical Judgment," Psychological Bulletin, 57, 116-131.

Huber, P. J. (1981), Robust Statistics. New York: John Wiley & Sons.

Johnson, Eric J., Robert J. Meyer, and Sanjoy Ghose (1989), "When Choice Models Fail: Compensatory Models in Negatively-Correlated Environments," Journal of Marketing Research, 26 (August), 255-270.

Levin, Lewis P., Jordan J. Louviere, Albert A. Schepanski, and Kent L. Norman (1983), "External Validity Tests of Laboratory Studies of Information Integration," Organizational Behavior and Human Performance, 31, 173-193.

Lutz, Richard J. (1989), "Presidential Address: Positivism, Naturalism, and Pluralism in Consumer Research: Paradigms in Paradise," in T. K. Srull (ed.), Advances in Consumer Research, Vol. XVI. Provo, UT: Association for Consumer Research, 1-8.

Moore, William L. and Morris B. Holbrook (1989), "Conjoint Analysis on Objects with Environmentally Correlated Attributes: The Questionable Importance of Experimental Design," Working Paper, University of Utah (June).

Rust, Roland T. (1988), "Flexible Regression," Journal of Marketing Research, 25 (February), 10-24.

Spence, Ian and Stephan Lewandowsky (1989), "Robust Multidimensional Scaling," Psychometrika, 54, 501-513.

----------------------------------------