The Role of Random Weights and Reliability in the Assessment of Multiattribute Attitude Models

Roger Best, University of Arizona
Del I. Hawkins, University of Oregon
Gerald Albaum, University of Oregon
ABSTRACT - In this study multiattribute attitude models using random numbers as weights performed as well as conventional attribute models. Furthermore, there was no relationship between model accuracy and the reliability of attribute beliefs and subject specified importance weights.
[ to cite ]:
Roger Best, Del I. Hawkins, and Gerald Albaum (1976) ,"The Role of Random Weights and Reliability in the Assessment of Multiattribute Attitude Models", in NA - Advances in Consumer Research Volume 03, eds. Beverlee B. Anderson, Cincinnati, OH : Association for Consumer Research, Pages: 88-91.

Advances in Consumer Research Volume 3, 1976      Pages 88-91

THE ROLE OF RANDOM WEIGHTS AND RELIABILITY IN THE ASSESSMENT OF MULTIATTRIBUTE ATTITUDE MODELS

Roger Best, University of Arizona

Del I. Hawkins, University of Oregon

Gerald Albaum, University of Oregon

ABSTRACT -

In this study multiattribute attitude models using random numbers as weights performed as well as conventional attribute models. Furthermore, there was no relationship between model accuracy and the reliability of attribute beliefs and subject specified importance weights.

INTRODUCTION

Since the studies reported by Bass (1972), Sheth (1972), and Talarzyk (1972), research on multiattribute attitude models has increased greatly. A review of some forty-two separate studies by Wilkie and Pessemier (1973) scrutinizes the state of this research and development. A principle issue in the evaluation of this research on multiattribute attitude medals has been the inclusion of differential weights (Bass & Wilkie, 1973).

The bulk of empirical evidence indicates that unit-weighted attribute models predict a criterion variable as well as, or slightly better than, subject-weighted models (Wilkie, 1973; Beckwith & Lehmann, 1973; Church-hill, 1972; Moinpour & MacLachlan, 1971; Scott & Bennett, 1971; Sheth, 1972). Yet many researchers have argued the necessity of differential weights in consumer attribute models (Wilkie, 1973; Beckwith & Lehmann, 1973; Myers & Gutman, 1974; Scott & Bennett, 1971). The purpose of this study is to analyze the impact of differential weights and measurement reliability in the assessment of multiattribute models of consumer attitudes.

SPECTRUM OF DIFFERENTIAL WEIGHTS

There exists a natural hierarchy of differential weights for the relationship between a criterion and set of predictor variables. At one end of the spectrum are least-squares derived weights which, when independent and stable, provide the most efficient weights for a set of attributes. Within the context of multiattribute models of consumer attitudes, Beckwith & Lehmann (1973) have demonstrated that least-squares derived weights yield models which predict brand preference better than attribute models using either subject weights or unit weights.

At the other end of the spectrum are differential weights selected from a table of random numbers. This method of weighting attribute models should produce the least effective set of attribute weights for predicting a given criterion. Between these two extremes lie subject-weighted and unit-weighted multiattribute attitude models.

Consumer attitude research is carried out most effectively at the individual level of analysis (Myers & Gutman, 1974). However, this presents a problem since sample sizes needed to achieve stable least-squares weights are difficult to achieve at this level of analysis. To achieve stability approximately 15 to 20 observations are needed for each attribute included in the model. Since most studies of attribute models involve sample sizes ranging from 5 to 10 observations, least-squares derived models do not offer a viable alternative for either applied usage or benchmarking the relative contribution of models incorporating either subject-stated or unit weights. However, at the opposite end of the spectrum, randomly chosen weights provide an alternative method of assessing model performance, a method that is independent of individual sample sizes. The question becomes: "How much better than models using random weights are the predictions of attribute models using the subject-stated or unit weights?"

METHODOLOGY

A sample of 130 adult female shoppers from four local church groups completed a questionnaire concerning shoppers' attitudes. Each group received a self-administered questionnaire with the stated purpose of measuring their "opinions and ideas about various department stores and the shopping process itself." An interviewer was present to answer any questions that the respondents had concerning how to complete the questionnaire.

Respondents' beliefs toward five department stores were measured on a 10-item, 6-interval Stapel scale (Hawkins, et al., 1974). The ten attributes (see Table 1) were chosen from a set used in previous research on department store images using the Stapel scale. The five department stores were selected based on the fact that they were individually well-known in the local community and collectively represented a range of store types. The stores included two high quality regional chains, two medium quality national chains, and one lower quality national chain.

TABLE 1

STORE ATTRIBUTES USED TO EVALUATE RESPONDENTS' BELIEFS TOWARD FIVE DIFFERENT STORES

Differential weights were obtained in a two step-process. Each respondent first listed the five attributes she felt were most important in her evaluation of the department stores. Then using a constant sum scale (Hughes 1971) respondents were instructed to allocate 100 points among the five attributes to indicate the relative importance of each attribute.

From each subject's responses, three attribute models were constructed: a subject-weighted model, a unit-weighted model, and a randomly-weighted model using weights drawn from a uniform distribution of random numbers varying from 0 to 100. Each of the multiattribute attitude models is shown below.

EQUATION

Where:

i = attribute or store characteristic

j = store

k = female shopper

such that:

"jk = shopper k's attitude score for store j

Iik = the importance weight given to attribute i by shopper k

Bijk = shopper k's belief as to the extent to which attribute i is offered by store j

Rik = the random weight given to attribute i for shopper k

Two measures were used as criterion variables. One criterion was simply the respondents' rank order of store preferences.

The second criterion was obtained from a constant sum scale by instructing the respondent to distribute 100 points among the five stores to indicate her relative purchases at these stores over the past year. The linear association between the criterion measures and the attitude scores produced by the three attribute models was compiled for each respondent using the Spearman-Rank Order Correlation coefficient.

A second questionnaire was mailed to each participant approximately ten days after the administration of the first questionnaire. The second questionnaire was identical to the first. Test-retest reliability measures of attribute ratings, subject-stated importance weights, rank order of store preferences and subject-stated store behavior were computed for each respondent using the Pearson-Product Moment Correlation coefficient.

ANALYSIS AND RESULTS

From the original sample, 114 questionnaires were usable in the analysis, while 70 usable questionnaires were obtained from the second (retest) questionnaire.

The rank order correlation coefficients computed to measure each respondents' linear association between attitude scores and store preferences are plotted for the entire sample in the form of a histogram for each model type specified in this study (see Figure 1). The correlation coefficient distributions produced from subject-stated and unit-weighted attribute models were compared to the distribution produced by the randomly-weighted attribute model using the two-tailed version of the Kolmogorov-Smirnov two sample test. In this case, there were no significant differences(a = .10) between the distributions generated by the alternative weighting schemes.

FIGURE 1

FREQUENCY DISTRIBUTIONS OF INDIVIDUAL CORRELATIONS FOR ATTITUDE MODELS USED TO PREDICT STORE PREFERENCE

The same procedure was followed using subject-stated behavior as the criterion. The distribution of rank order correlation coefficients for each model type is shown in Figure 2. Again, no significant differences were detected at the .10 level when comparing the distributions generated with subject-stated and unit weights with the distribution produced using random numbers as weights. The average correlation between the two criterion variables, store preference and stated shopping behavior was .83 (p<.10).

The reliability of the data used to construct these models was assessed with individual measures of test-retest reliability. The test-retest reliability of belief ratings was computed for each respondent and is displayed in the form of a histogram in Figure 3. The average correlation between belief ratings provided in the test and retest questionnaires was .53 (significant at p<.10). A similar distribution is provided for the subject-stated weights in Figure 4. In the case of subject-stated importance weights the average test-re-test correlation was .55 which was not significant at p=.10. The average test-retest correlations for store preferences and stated shopping behavior were . 78 and .80 respectively.

FIGURE 2

FREQUENCY DISTRIBUTION OF INDIVIDUAL CORRELATIONS FOR ATTITUDE MODELS USED TO PREDICT STATED STORE BEHAVIOR

FIGURE 3

FREQUENCY DISTRIBUTION OF INDIVIDUAL TEST-RETEST COP. RELATIONS

FIGURE 4

FREQUENCY DISTRIBUTION: INDIVIDUAL RELIABILITY OF STATED IMPORTANCE WEIGHTS

Table 2 was constructed to depict the linear association between model accuracy and test-retest reliability. For each of the model-types shown in Table 2, there was no relationship (a=.10) between the model accuracy (correlation between attitude scores and store preferences) and reliability of belief ratings or subject-stated importance weights.

CONCLUSIONS

In the aggregate, distributions of rank order correlation coefficients generated by subject-stated and unit-weighted attitude models did not differ significantly from a distribution of correlations derived from models weighted with random numbers. In addition, the reliability of belief ratings and subject estimated importance weights had no impact on the linear association between attitude scores produced by the three model-types and store preferences.

TABLE 2

CORRELATIONS BETWEEN MODEL ACCURACY AND BELIEF AND WEIGHT TEST-RETEST RELIABILITY

While this study dealt with beliefs toward stores, Holmes (1974) found approximately the same level of test-retest reliability for beliefs related to a set of brands of beer. Thus, while one might expect the reliability of belief ratings to be lower for stores because of departmentized effect on attributes (attributes like service and friendliness could differ greatly between departments within the same store), belief reliability in this study was as high as that reported for a set of products.

The purpose of this study was to evaluate the impact of differential weights and measurement reliability on the predictive accuracy of multiattribute attitude models.

The analysis produced the following results:

1. Randomly weighted multiattribute attitude models perform as well as subject-weighted or unit-weighted models.

2. The accuracy achieved by multiattribute attitude models is independent of the reliability of belief ratings and subject-specified weights which are essential components of the model.

3. A simple rank order measure of store preference is a substantially better predictor of stated shopping behavior than any of the multiattribute models developed in this study.

These findings lead to the general conclusion that multiattribute models are suspect both in terms of their general predictive ability and their ability to aid in our understanding of the variables that influence consumer decisions.

REFERENCES

Frank M. Bass, "An Attitude Model for the Study of Brand Preference," Journal of Marketing Research, 9 (February, 1972), 93-6.

Frank M. Bass, "Fishbein and Brand Preference: A Reply," Journal of Marketing Research, 9 (November, 1972), 461.

Frank M. Bass and William L. Wilkie, "A Comparative Analysis of Attitudinal Predictions of Brand Preference," Journal of Marketing Research, 10(August, 1973), 262-9.

Nell E. Beck-with and Donald R. Lehmann, "The Importance of Differential Weights in Multiple Attitude Models of Consumer Attitude," Journal of Marketing Research, 10 (May, 1973), 141-5.

Gilbert A. Churchill, "Linear Attitude Models: A Study of Predictive Ability," Journal of Marketing Research, 11(August, 1974), 428-6.

Del I. Hawkins, Gerald Albaum, and Roger Best, "Stapel Scales or Semantic Differential in Marketing Research?" Journal of Marketing Research, 11(August, 1974), 318-22.

David G. Hughes, Attitude Measurement for Marketing Strategies,(Glenview, Illinois: Scott-Foresman, 1971).

Reza Moinpour and Douglas L. MacLachlan, "The Relation Among Attribute and Importance Components of Rosenburg-Fishbein Type Attitude Models: An Empirical Investigation,'' Proceedings, Fall Conference, American Marketing Association, 1971, 365-75.

James H. Myers and Jonathan Gutman, "Validating Multi-attribute Attitude Models," Proceedings, Fall Conference, American Marketing Association, 1974.

J. C. Nunnally, Jr., Introduction to Psychological Measurement (New York: McGraw-Hill Book Company, 1973).

Jerome E. Scott and Peter D. Bennett, "Cognitive Models of Attitude Structure: Value Importance is Important," Proceedings, Fall Conference, American Marketing Association, 1971, 346-50.

Jagdish G. Sheth, "Perceived Instrumentality and Value Importance as Determinants of Attitudes," Journal of Marketing Research, 9(February, 1972), 6-9.

Jagdish G. Sheth, "Reply to Comments on the Nature and Uses of Expectancy-Value Models in Consumer Attitude Research," Journal of Marketing Research, 9(November, 1972), 462-5.

Wayne W. Talarzyk, "A Reply to the Response to Bass, Talarzyk, and Sheth," Journal of Marketing Research, 9(November, 1972), 465-7.

William L. Wilkie and Edgar A. Pessemier, "Issues in Marketing's Use of Multiattribute Attitude Models," Journal of Marketing Research, 10(November, 1973), 428-41.

----------------------------------------