Multiattribute Scaling Models: Some Observations

Naresh K. Malhotra, Georgia Institute of Technology
ABSTRACT - Some observations are made on four issues which deserve more attention in conjoint analysis and multidimensional scaling research. These issues concern the use of metric vs. nonmetric techniques, robustness of multiattribute scaling models, cognitive processes underlying these models and the role of individual variables.
[ to cite ]:
Naresh K. Malhotra (1981) ,"Multiattribute Scaling Models: Some Observations", in NA - Advances in Consumer Research Volume 08, eds. Kent B. Monroe, Ann Abor, MI : Association for Consumer Research, Pages: 306-308.

Advances in Consumer Research Volume 8, 1981      Pages 306-308

MULTIATTRIBUTE SCALING MODELS: SOME OBSERVATIONS

Naresh K. Malhotra, Georgia Institute of Technology

ABSTRACT -

Some observations are made on four issues which deserve more attention in conjoint analysis and multidimensional scaling research. These issues concern the use of metric vs. nonmetric techniques, robustness of multiattribute scaling models, cognitive processes underlying these models and the role of individual variables.

INTRODUCTION

Over the last decade multidimensional scaling and conjoint measurement have become extremely popular for assessing consumers' perceptions and preferences. Due to their popularity, the multiattribute scaling models have also filtered into the annual conferences of the Association for Consumer Research. The last several conferences have featured papers and/or sessions on this topic. Hence, it was only appropriate that the current conference also include a session on this important area.

However, several issues concerning the multiattribute scaling models remain unresolved or even virtually untouched. Some observations are offered on four such issues: the use of metric vs. nonmetric estimation procedures, the robustness of multiattribute scaling models, the cognitive processes underlying these models and individual differences impinging on these models. No attempt is made to offer comprehensive reviews on each of these issues. Rather, the observations made point to, in the opinion of the author, the crux of the issues involved. It is possible that some of the author's views may not be shared by others working in this area. Also, the choice of these specific issues was influenced more by the author's familiarity and experience with then and does not necessarily reflect their intrinsic importance. Some of these issues have also been dealt with in the papers presented in this session.

METRIC VS. NONMETRIC

The issue of concern is whether the nonmetric techniques are generally superior to the metric procedures for estimating parameters in multiattribute scaling models. The nonmetric techniques assume that the dependent variable or input data is ordinally scaled or weaker. The metric procedures, on the other hand, are based on the assumption that the dependent variable or input data is at least intervally scaled. A brief discussion of the popular metric and nonmetric estimation procedures for conjoint analysis may be found in Green and Srinivasan (1978). An excellent classification of alternative metric and non- metric multidimensional scaling procedures for analyzing perceptions and preference data is provided by Green and Rao (1972, p. 13). In the context of conjoint analysis, simulation studies (Carmone, Green and Jain 1978; Cattin and Wittink 1976) as well as empirical studies (Jain et el. 1979; McCullough 1978; Montgomery, Wittink and Glaze 1977) have found that the metric OLS regression does as well as the other nonmetric estimation procedures. In the case of multidimensional scaling also, the nonmetric and metric techniques yield very similar results. This is particularly true if the scaling has been done in correct dimensionality (Green 1975). Moreover, certain flexibilities offered by the nonmetric procedures such as the use of alternative distance functions have turned out to be more of illusory benefits. The recent findings by Hauser and Koppelman (1979) also should make us pause and take a hard look at the nonmetric multidimensional scaling techniques.

In terms of user-oriented criteria such as algorithm availability, adaptability in the user's system, extent of pre-processing of data, and computing costs, the simpler metric procedures may offer some relative advantages (Jain et al. 1979). Further empirical evidence on the relative performance of metric and nonmetric procedures is needed. However, at this stage, it is reasonable to conclude that the general superiority of the more complex nonmetric procedures over the simpler metric procedures has not been so far demonstrated. It is true that in certain situations, particularly where the input data should be treated as only ordinal, the use of nonmetric techniques would be more appropriate. However, consumer researchers must safeguard against the tendency to automatically employ nonmetric procedures in every situation just because they appear to be more sophisticated.

ROBUSTNESS

The various issues related to reliability and validity of conjoint analysis have been dealt well by Green and Srinivasan (1978). This discussion would focus on only the robustness of conjoint analysis and multidimensional scaling solutions with respect to incomplete or missing data. As the percentage of data (based on all possible observations) obtained from the respondents increases, the reliability and stability of the estimated coefficients should increase and the error in prediction should decline. On the other hand, obtaining large amount of data from the respondent in a given unit of time could cause fatigue and information overload.

Careens, Green end Jain (1978) undertook a Monte Carlo simulation to examine the effect of missing data on pert worth recovery in conjoint analysis. They varied the number of observations at four levels 18, 27, 54, and 243 (full factorial). Thus, the percentage of missing data was varied from 92.6% to 0%. Their results indicated that the part worths recovery was almost as good with 18 observations as with the full set of 24&3 observations. In a recent empirical investigation, Leigh, Mackay and Summers (1981) investigated the relative performance of full, half and quarter factorial designs. Their study suggests that the less fractionated design tends to produce higher test-retest part worth reliabilities when the number of attributes is limited. Of course, with a larger number of attributes and levels, the less fractionated designs become impractical and the researcher may be forced to adopt the more fractionated designs accommodations a higher percentage of missies data.

In the context of multidimensional scaling, the empirical study by Jain, Malhotra and Mahajan (1978) revealed that configuration recovery because significantly poorer as the percentile of missing data increased iron 20% to 40% to 60%. However, the effect of employing cyclical designs or random method of selecting judgmental pairs on which information was obtained was not found to be significant. Similar results were obtained in the Monte Carlo simulation performed by Spence and Domoney (1974).

In sum, it seems that the strategy of collecting incomplete data to minimize respondent task involves trade-offs. While the collection of incomplete data from the subjects does reduce respondent fatigue, information overload and costs of data collection, it may not adequately capture the underlying perceptions and preferences. However, the question of how such trade-offs are to be made is far from settled. Further empirical research is needed in this area.

COGNITIVE PROCESSES

The cognitive processes underlying the formation of consumer's perceptions and preferences have not received due attention from marketing researchers employing conjoint analysis end multidimensional scaling techniques. The major issue here is whether and to whet extent the model assumed to estimate parameters represents the actual decision process employed by the individual. In conjoint analysis, for example, a particular decision rule, such as the additive or interactive, is assumed to estimate the parameters of the individual's preference function. This has generally been done without adequate regard to the actual choice rule employed by the respondent. Them, post estimation attempts are made to justify the model used as Being appropriate. The additive model has been the most popular. The use of a goodness or badness of fit measure, for example Kruskal's stress (1965), as well as the prediction criteria, to determine the appropriateness of the linear model has been rather misleading. Studies have shown that the additive model can yield good fits even when the data corresponds to other decision models (Dawes and Corrigan 1974; Green 1968). Furthermore, the correlational methods used to assess predictive ability suffer from several problems which limit the usefulness of these methods for examining the appropriateness of alternative decision rules. A discussion of the problems involved may be found in Bettman (1979, pp. 191-193).

An algorithmic development which somewhat alleviates this problem is the multistage decision process model by Srinivasan (1977). This is a general model which captures many of the decision rules considered in information processing literature. By taking a multistage view of the consumer decision process, Srinivasan's model takes us a step closer to modeling the actual choice process.

While past attempts have been lacking in this respect, future use of multiattribute scaling models should attempt to formulate estimation models based on the actual decision process adopted by the individual. An approach to accomplish this has been advocated by Olshavsky and Acito (1980).

INDIVIDUAL VARIABLES

The formation of perceptions and preferences is affected by individual differences (Bettman 1979). Simplistic as this statement may seen, conjoint analysis and multidimensional scaling analysis have, hithertofore, not directly focused on such differences. Individual differences related to information processing would seen to be natural candidates for investigation. Of particular import are consumer cognitive styles (Goldstein and Blackman 1978).

Consider, for illustration, the particular cognitive style of cognitive differentiation. Differentiation refers to the number of dimensions used by an individual in processing information (Bieri 1971). The more cognitively complex individual is assumed to have available a greater number of dimensions with which to construe his environment. Thus, in the context of multidimensional scaling, it is reasonable to expect that for the cognitively complex individuals, a higher dimensionality space mould be required to adequately represent their perceptions. As individuals with high cognitive complexity form their perceptions in complex ways, a greater percentage of data (in the context of incomplete data) may be required to capture their perceptions and preferences. Likewise, the decision rules used by individuals are also likely to vary depending upon the cognitive complexity level. In similar vein, a number of other interesting hypotheses may be generated. Thus, the need to incorporate individual differences in multidimensional scaling and conjoint analysis of consumer perceptions and preference is apparent, and, to date, largely unmet.

SESSION ON MULTIATTRIBUTE SCALING MODELS

Two of the three papers in this session deal with some of the issues which have been raised in the foregoing discussion. The paper by Green and DeSarbo describes and applies the models for representing unconstrained choice data in a multidimensional space. Acito and Olshavsky examine the robustness of conjoint analysis with respect to the levels per attribute used to construct the stimuli set. A protocol analysis of the decision rules employed by the respondents is also conducted. Leigh, Mackay and Summers examine the robustness of conjoint analysis with respect to the percentage of missing data and measurement scale for the dependent variable.

REFERENCES

Bettman, J. R. (1979), An Information Processing Theory of Consumer Choice, Reading, MA: Addison-Wesley.

Bieri, J. (1971), "Cognitive Structures in Personality," in Personality Theory and Information Processing, H. M. Schroder and P. Suedfeld, eds., New York: The Ronald Press Co., 178-308.

Carmone, F. J., Green, P. E., and Jain, A. K. (1978), "The Robustness of Conjoint Analysis: Some Monte Carlo Results," Journal of Marketing Research, 15, 300-3.

Cattin, P. and Wittink, D. R. (1976), "A Monte Carlo Study of Metric and Nonmetric Estimation Methods for Multiattribute Models," Research Paper No. 34l, Graduate School of Business, Stanford University.

Dawes, R. N. and Corrigan B. (197&), "Linear Models in Decision Making," Psychological Bulletin 81 (February) 95-106.

Green, B. F. (1968), 'Descriptions end Explanations: A Comment on Papers by Hoffman and Edwards," in Formal Representation of Human Judgement, Benjamin Kleinmuntz, ed., New York: Wiley, 91-98.

Green, Paul E. (1975), 'Marketing Applications of MDS: Assessment and Outlook," Journal of Marketing, 39, 24-31.

Green, Paul E. and Ran, V. R. (1972), Applied Multidimensional Scaling, New York: Holt, Rinehart and Winston, Inc.

Green, Paul E. and Srinivasan, V. (1978), "Conjoint Analysis in Consumer Research: Issues and Outlook," Journal of Consumer Research, 5 (September), 103-123.

Goldstein, K. N. and Blackman, S. (1978), Cognitive Style: Five Approaches and Relevant Research, New York: John Wiley and Sons.

Hauser, J. R. and Koppelman, F. S. (1979), "Alternative Perceptual Mapping Techniques: Relative Accuracy and Usefulness," Journal of Marketing Research, 16, 495-506.

Jain, A. K., Acito, F., Malhotra, N. K., and Mahajan V. (1979), "A Comparison of the Internal Validity of Alternative Parameter Estimation Methods in Decomposition Multiattribute Preference Models," Journal of Marketing Research, 16, 313-323.

Jain, A. K., Malhotra, Naresh K. and Mahajan, Vijay (1978), "The Effect of Missing Values in Similarity Judgments on configuration Recovery in MDS Models: An Experimental Examination," paper presented at the First Joint Meeting of the Psychometric Society and The Society for Mathematical Psychology, McMaster University, Hamilton, Canada, August 25-27.

Kruskal, J. B. (1965), "Analysis of Factorial Experiments by Estimating Monotone Transformation of the Data," Journal of the Royal Statistical Society, Series B (March), 251-63.

Leigh, T. W., MacKay, D. B., and Summers, J. O. (1981), "An Alternative Experimental Methods for Conjoint Analysis," Advances in Consumer Research, 8 (forthcoming).

McCullough, J. M. (1978), "Identification of Preference Through Conjoint Measurement: A Comparison of Data Collection and Analytical Procedures," Working Paper, College of Business Administration, The University of Arizona.

Montgomery, D. B., Wittink, D. R. and Glaze, T. (1977), "A Predictive Test of Individual Level Concept Evaluation and Trade-Off Analysis," Research Paper No. 415, Graduate School of Business, Stanford University.

Olshavsky, P. W. and Acito, F. (1980), "An Information Processing Probe into Conjoint Analysis," Decision Sciences, 11 (July), 451-470.

Spence, I. and Domoney, D. W. (1974), "Single Subject Incomplete Designs for Nonmetric Multidimensional Scaling," Psychometrika, 39, 469-490.

Srinivasan, V. (1977), "An Approach to the Modeling and Estimation of Consumer Multi-Stage Decision Processes," Paper presented at the AMA/MSI/PIT Conference on Analytical Approaches to Product and Marketing Planning, Pittsburgh, Pennsylvania, November.

----------------------------------------