A Search For Models of Consumers Union's Brand Evaluation: a Multidimensional Approach

Citation:

Vithala R. Rao and G. David Hughes (1971) ,"A Search For Models of Consumers Union's Brand Evaluation: a Multidimensional Approach", in SV - Proceedings of the Second Annual Conference of the Association for Consumer Research, eds. David M. Gardner, College Park, MD : Association for Consumer Research, Pages: 164-174.

Proceedings of the Second Annual Conference of the Association for Consumer Research, 1971     Pages 164-174

A SEARCH FOR MODELS OF CONSUMERS UNION'S BRAND EVALUATION: A MULTIDIMENSIONAL APPROACH

Vithala R. Rao, Cornell University

G. David Hughes, Cornell University

[The authors acknowledge the generous support for computing funds from the Office of Sponsored Research, Cornell University and thank Messrs. Jorge Doehner, Frank Amthor and Geoffry Soutar for their assistance in data compilation and analysis.]

[Respectively Assistant Professor and Associate Professor at the Graduate School of Business and Public Administration, Cornell University, Ithaca, New York.]

An ever increasing technology complicates the consumer buying decision. The recent growth in consumerism reflects the consumer's need for reliable and valid information to enable him to maximize his well being in a complex society. The consumer has five sources of information when making a buying decision: personal (experience), commercial (manufacturer ant retailer promotion), interpersonal (opinion leaders, neighbors, and friends), government, and independent consumer rating services. While consumer rating services have met part of the demand for technological information they have not fulfilled their potential because they to not report the model they use to evaluate brands. Furthermore, their reports are rendered obsolete by model changes in the brands evaluated by them.

Without the evaluation model employed by the rating services such as the Consumers Union the consumer cannot integrate the information from all of these sources, thereby making a decision to meet his personal preferences. He is implicitly led to accept the judgment of the rating services in their selection and weighting of attributes. He must postpone his purchase until the services evaluate new products or he must forego the advantages of new products and purchase those which have been rated previously. He cannot evaluate objectively brands not covered by the services, such as regional brands. In short, failure to disclose fully the brand evaluation model limits the consumer's ability to be a better shopper. Furthermore, it benefits the manufacturer who is able to reconstruct it from reported data thereby inhibiting competition through meaningful product improvements. Thus, a policy of less than full disclosure is unfair to the consumer and some manufacturers.

The purpose of this paper is to demonstrate how recently developed behavioral analytical techniques (nonmetric multidimensional scaling) and multivariate statistical techniques may be used to search for brand evaluation models that approximate those wed by one rating service, Consumers Union (CU), using only the data reported in its publication, The Consumer Reports.

The policy implications seem clear. Rating services should practice full disclosure by reporting their brand evaluation models. Furthermore, the form of this reporting should enable the consumer to add attributes and change the weights of attributes so that his purchases will meet his idiosyncratic preferential considerations.

The paper is organized into four major parts: (a) the data base, (b) a description of the methodology, (c) approximating models, and (d) policy implications and suggestions for future research.

DATA BASE

The data for this search were collected from the Consumer Reports of the CU for the years 1967 and 1968. In all, nine product classes were studied. The information published by CU for any product class includes: (a) brand profiles on product attributes; (b) list price; (c) an overall quality evaluation of brands into ordered categories such as 'Acceptable-Excellent,' 'Acceptable-Very Good,' etc.; (d) check rating for selected brands indicating that they are qualitatively outstanding; and (e) indication of 'Best Buys' (i.e., the highest quality per unit price) wherever available. Items (a), (c), and (t) were used to develop models of overall brand evaluation and check ratings. The analysis was confined to those brands for which complete profile information was reported. The details on the number of brands, number of attributes (excluding price) and the source of data employed in this study are shown in Table 1. It was not possible to use all of the brand information within product categories. In some categories brands are ranked according to overall quality, but they are listed alphabetically in other categories. Data were incomplete for brands which were judged not acceptable.

TABLE 1

DESCRIPTION OF DATA USED

METHOD

The analysis procedure for each product class consisted of three steps summarized in Figure 1 as follows: (a) reduce attribute space, (b) develop a model of overall brand evaluations, and (c) develop a model of check rating.

Reduce Attribute Space

To enable generalizations across product classes the original attributes reported by CU were reduced to a three-dimensional space. [This space reduction enables one to conserve degrees of freedom particularly for those product classes with few brands.] For each product class, a matrix of interbrand distances was computed using standardized brand profiles. These distances were analyzed using TORSCA, the method of multidimensional scaling (Young and Torgerson, 1967) to yield reduced configurations in three dimensions. The three dimensional configurations were interpreted using judgment (extreme analysis) and the property fitting procedure of PROFIT (Chang and Carroll, 1970). These steps are shown in the left panel of Figure 1. These three dimensions served as the predictor variables for the models.

FIGURE 1

ANALYSIS STEPS

Develop a Model of Overall Brand Evaluations

The model approximating the overall brand evaluations was developed utilizing the multidimensional generalization of Coombsian unfolding model to depict preferential data (Carroll and Chang, 1967). The nonmetric version [The block monotone option which handles ties in the evaluation vector was chosen for this analysis.] of Carroll and Chang's algorithm, known as PREFMAP, was used to portray the ordered class vector of brand evaluations in the three dimensional reduced configuration developed above (see center panel of Figure 1). The vector model, which fitted the data better than the point model, is as follows:

f(R_i) - b₀ + b₁ X_1i + b₂ X_2i + b₃ X_3i + error

i = 1, 2,..,,n

where

n = number of brands;

R_i = overall rating in the ordered class for the brand i;

f(R_i) = monotone transformation of the rating, Ri;

b_j = importance assigned to the jth dimension, j = 1, 2, 3;

X_ji = coordinate value for the ith brand on the jth dimension in the reduced space; and

b₀ = constant term.

Develop a Model of Check Rating

A two-group discriminant analysis was used to develop a model to approximate CU's check rating of brands. The three dimensions of the reduced space were used as three variables in a step-wise discriminant analysis (Anderson, 1958 and Dixon, 1968) to classify the brands into two groups = check rated and others. This model in symbols is:

Check rate the brand i if d₀ + d₁X_1i + d₂X_2i + d₃X_3i > C and do not check rate otherwise,

where values of d's are estimated and C is a critical level for the discriminant function. The standardized values of d₁, d₂ and d₃ reflect the implicit importances assigned to the dimensions. Goodness of fit of this model is measured by the percentage of brands correctly classified as check rated and not. This phase of the analysis is shown in the right Panel of Figure 1.

RESULTS

The three steps described above were applied to each of the nine product classes (Table 1). Because of the extensive findings generated by these steps, the results are reported only in a summary form for all products.

Dimensionality of Reduced Space

The values of stress [Stress, defined by Kruskal (1964), is a measure of badness of fit. The lower the stress the better is the fit. Stress is a decreasing function of number of dimensions. No statistical tests are available for judging dimensionality.] for 3 and 2 dimensions obtained in TORSCA analysis (stage 2) of interbrand distances are presented in Table 2. No systematic differences in the degrees of fit can be noticed between the product classes. In the light of significantly better fits and using the Monte Carlo tables developed by Klahr (1969), the three dimensional solutions are utilized for further analysis.

TABLE 2

STRESS BY DIMENSIONALITY FOR ALL PRODUCT CLASSES

Dimensional InterPretation

The reduced spaces were interpreted by fitting the various attributes to the reduced space using the linear version of the PROFIT algorithm. This analysis was augmented by comparing the attributes of brands at the extremes of each dimension.

The three dimensions can be interpreted as technical complexity, performance, and user convenience. The order in which these dimensions appear (according to the variance accounted for) varies across the product classes, as shown in Table 3. These dimensions seem to be consistent with CU's reported policy of judging brands according to mechanical and functional characteristics rather than aesthetic ones. The construct of 'technical complexity' is what is built into the product by the manufacturer, while the 'performance' construct represents the result of the engineering design aspects under normal operating conditions. The third construct of 'user convenience' represents how conveniently a consumer can interact with the machine. The nine products examined lead one to infer that CU places greater emphasis on the technical aspects of the product than on convenience in use.

TABLE 3

DIMENSIONAL INTERPRETATION FOR ALL PRODUCT CLASSES

Approximating Model for Overall Brand Evaluation

The results of fitting the vector model of PREFMAP to the brand evaluations for the nine product classes are summarized in Table 4. The high multiple correlation coefficients indicate that this linear compensatory model is a good representation of the overall brand evaluations of CU.

TABLE 4

DEGREE OF FIT AND DIRECTION COSINES FOR MODEL APPROXIMATING OVERALL EVALUATIONS USING PREFMAP

In the light of the positive coefficients, brands falling away from the origin in the direction of the vector represented by the direction cosines in Table 4 are judged better than those nearer to the origin. All planes perpendicular are, ln fact, the indifference planes in the three-dimensional space.

While the single functional form fits all nine products well, the direction cosines vary greatly among products. The nine models can be summarized graphically, as shown in Figure 2, where each plotted point represents a product class. The finding that product classes are distributed in this space suggests several hypotheses regarding CU's implicit evaluation model. First, weights may vary systematically according to predefined product categories. Second, there may be individual differences in values among the judges which are not resolved prior to reporting. Finally, weights are changed throughout product life cycles. For instance, as brands become homogeneous with regard to technical complexity this dimension is given less weight.

FIGURE 2

SUMMARY REPRESENTATION OF OVERALL BRAND EVALUATIONS FOR NINE PRODUCT CLASSES

The first and third of these hypotheses were examined within the limits of available data. For the first hypothesis, clusters of products were developed using the direction cosines (Table 4) and the hierarchical cluster method according to Howard and Harris (1966). The composition of two, three, and four clusters is shown in Table 5. Because no common characteristic could be identified for these products, no interpretation is possible. For example, at the two-cluster level, the products of color TV, monochrome TV, open electric broilers, phono cartridges, and upright vacuum cleaners form one cluster with no obvious common characteristic.

TABLE 5

HIERARCHICAL CLUSTERING OF NINE PRODUCTS

However, there is support for saying that the weights assigned to technical complexity and performance dimensions vary among product classes according to the stage of the life cycle of the product. For example, Color TV Sets receive a higher weight on the technical complexity dimension than do Monochrome TV Sets. The reverse is true for the performance dimension. Similar patterns can be observed between the product classes of Electric Broilers Cabinet (a recent introduction) versus Open (an established product); and Vacuum Cleaners, Canister (a recent introduction) and Upright (established product). This implies that Consumers Union tends to rely more heavily on the engineering design considerations for newer products and less 80 for the established ones. This may be due to the fact that brands in established product categories are less differentiated since, over time, several manufacturers can acquire comparable technological competence.

Approximating Model for Check Rating

The three dimensions of the reduced space were used as predictors in developing a discriminant function for the check rating procedure adopted by CU. The proportion of brands correctly classified by the function and the standardized coefficients are presented in Table 6. In four out of nine product classes the model correctly classifies all brands, The average across all products is 90.9 percent. Generally, the technical complexity and performance dimensions turn out to be important variables in the discriminant functions. These dimensions are not assigned uniform importance, thereby indicating the possibility of individual differences in CU's judging procedures.

TABLE 6

RESULTS OF DISCRIMINANT ANALYSIS

Summary

The various results may now be summarized:

1. CU's measurements of brand profiles can be reduced to a three dimensional model for each of the nine products. These dimensions are technical complexity, performance and user convenience.

2. In the similarity spaces of brands, the potency of the dimensions varies across product classes.

3. The generalized Coombsian model of vector representation portrays the overall brand evaluations extremely well for all product classes. Thus, the model employed by CU for overall evaluations appears to be compensatory in nature.

4. The importance assigned to the three dimensions varies across product classes. This variation could not be attributed to any characteristics inherent in the product classes. There appear to be individual CU expert differences in evaluating brands of several product classes.

5. The linear discriminant function, developed using the three reduced dimensions as predictors, classifies the brands well into check rated and not check rated. The proportion correctly classified is over 90 percent on the average.

6. The variables that turn out to be significant in the discriminant functions differ across products, again pointing to the possibility of individual expert differences at Consumers Union.

POLICY IMPLICATIONS

Several implications for policy formulation emerge from this study, not only for Consumers Union but also for various manufacturers and government.

With regard to Consumers Union, there appears to be a clear need for appraising its current system of testing brands and reporting brand information. First, evidence suggests that the number of attributes selected for testing be reduced, thereby lowering the overall costs of Product testing.

Second, CU might consider using scales with more response categories for rating attributes as opposed to the grosser scales now in vogue. This practice should be valuable in better describing the similarity structure of various brands (Green and Rao, 1970). Obviously, the need exists for adhering to a uniform format for reporting brand profiles and final evaluations. The current CU practice of not reporting comparable and complete information for those brands judged as "not acceptable" does limit an objective consumer in examining for himself the reasoning behind CU's evaluations.

Furthermore, CU might benefit from a reappraisal of its differential weighting system so implicit in its evaluations across different product categories. If in fact individual expert differences do exist, it is in CU's own interest to examine and resolve them before passing on its recommendations to an ordinary consumer. Interests of all concerned would be better served if CU were to reveal the full details of its current policy of brand recommendations, particularly in light of implicit lack of uniformity across products. The report on any product should further enable a consumer to incorporate his prior knowledge of product class, additional sources of information, additional attributes and information about new brands that might have appeared in the market-place since the issuance of CU's report. It should also be flexible enough to enable the consumer to apply his own idiosyncratic weighting procedures.

From the manufacturer's point of view, this study should assist in designing product improvement strategies. Manufacturers would benefit by developing models for assessing the effect of a given CU ranking of their brands on market share, for predicting how CU might rank their brands and by utilizing such information in allocating their marketing effort.

One significant public policy implication lies in formulating regulations for full disclosure of the methods adopted by product rating services such as CU. There appears to be no reason for these services to be exempt from policies of full disclosure.

FUTURE RESEARCH

Various researchable questions emerge from the foregoing analysis in the general areas of consumer research and public policy. Some of these are enumerated below with a view to fostering interest of various researchers and policy makers.

1. How do individual value systems influence transformation of objective space into perceptual judgments?

2. Is there any evidence that manufacturers use Consumer RePorts for product changes?

3. Do the findings extend themselves to other product categories? If so, is there any possibility for reaching a general statement of product evaluations of rating services?

4. Can the life cycle hypothesis be validated by an independent set of CU's evaluation data?

5. Does the previously developed model of brand evaluations in any product class predict evaluations for new brands or improved brands?

6. Can methods be developed to assist CU experts in resolving their value differences, if they exist? Several experiments and group training sessions may be designed based on small group behavioral theories.

7. Are there other kinds of models that could portray CU judgments? For example, the CU experts might be following a lexicographic model wherein different attributes of brands are ranked and brands are classified with prespecified threshold levels.

8. Can models be developed for predicting the best buy brand? In particular, can the interval scaled measures for final evaluations provided by the measurement models of preference be used to predict maximum quality per unit price (i.e., Best Buy)? It appears that this approach will have applications also in the area of unit pricing.

REFERENCES

Anderson, T. W. An Introduction to Multivariate Analysis. New York: Wiley, 1958.

Carroll, J. D. and J. J. Chang. Relating Preference Data to Multidimensional Scaling Solutions via a Generalization of Coombs' Unfolding Model. Bell Telephone Laboratories, Murray Hill, New Jersey, 1967 (mimeographed).

Chang, J. J. and J. D. Carroll. How to Use PROFIT, A Computer Program for Property Fitting, by Optimizing Nonlinear or Linear Correlation. Bell Telephone Laboratories, Murray Hill, New Jersey, 1970 (mimeographed).

Dixon, W. J. (ed.). Biomedical Computer Programs. Education and Health Sciences Faculty, University of California, Los Angeles, Second Edition, 1968.

Green, P. E. and V. R. Rao. Rating Scales and Information Recovery - How Many Scales and Response Categories to Use? Journal of Marketing, 1970, 34, 33-39.

Howard, Nigel and Britt Harris. A Hierarchical Grouping Routine, IBM 360/65 F0RTRAN IV Program. University of Pennsylvania Computer Center, Philadelphia, Pennsylvania, October 1966.

Klahr, David. A Monte Carlo Investigation of the Statistical Significance of Kruskal's Nonmetric Scaling Procedure. Psychometrika, 1969, 34, 319-333.

Kruskal, J. B. Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis. Psychometrika, 1964, 29, 1-27.

Young, F. W. and W. S. Torgerson. TORSCA, a FORTRAN IV Program for Shepard-Kruskal Multidimensional Scaling Analysis. Behavioral Science. 1967. 2, 498.

----------------------------------------