Product Class Experience, Dimensionality and Reliability: Their Relationship in a Nonmetric Scaling Study

Citation:

Adrian B. Ryans and Terry Deutscher (1975) ,"Product Class Experience, Dimensionality and Reliability: Their Relationship in a Nonmetric Scaling Study", in NA - Advances in Consumer Research Volume 02, eds. Mary Jane Schlinger, Ann Abor, MI : Association for Consumer Research, Pages: 285-294.

Advances in Consumer Research Volume 2, 1975      Pages 285-294

PRODUCT CLASS EXPERIENCE, DIMENSIONALITY AND RELIABILITY: THEIR RELATIONSHIP IN A NONMETRIC SCALING STUDY

Adrian B. Ryans, School of Business Administration, The University of Western Ontario

Terry Deutscher, College of Administrative Studies, The Ohio State University

[This research was supported in part by grants from the Marketing Science Institute and the Associates Research Fund, School of Business Administration, The University of Western Ontario.]

A method of estimating the number of dimensions a subject uses in making interbrand comparisons is presented. Using data from an experimental study on an inexpensive consumer durable, the relationship between an individual's product class experience and the number of dimensions used is examined. It is hypothesized that dimensionality and product experience are related in the following manner: a) individuals who received the product as a gift (i.e., they have usage experience but no shopping experience) use the fewest dimensions: b) individuals who have never owned the product (i.e., they have no usage experience and no shopping experience) use the most dimensions; and c) individuals who bought the product themselves (i.e., they have both shopping and usage experience) use an intermediate number of dimensions. It is further hypothesized that reliability in interbrand comparisons is inversely related to the complexity of the individual's perceptual structure.

INTRODUCTION

Until the last two or three years, the task of determining the underlying dimensionality of a set of similarity data that had been submitted to a nonmetric multidimensional scaling program was a very subjective one. Recently a number of approaches to assist in this task (Wagenaar and Padmos, 1970; Isaac and Poor, 1974; Spence, 1972) have been proposed. A closely related but more important, question from the viewpoint of a marketing researcher using nonmetric multidimensional scaling is "What factors influence the number of dimensions a subject will use in making similarity judgments?" Previous researchers (Wilkie and Weinrich, 1972; Wilkie and Pessemier, 1973) have provided evidence of individual differences among consumers in the number of dimensions they use in making interbrand comparisons within a product class. One might further hypothesize that the people using the most dimensions ceteris paribus will be the least reliable in making their similarity judgments, because they have the most complicated frame of reference, and, consequently, the greatest chance of altering their basis of comparison from one point in time to another (Schroder, Driver, and Streufert, 1967). This also is an important factor that must be considered by the marketing researcher.

The objectives of the study reported here are to develop specific operational hypotheses on these questions and to test these hypotheses in the context of a particular nonmetric multidimensional scaling study undertaken by one of the authors. Following the analysis will be a discussion of the implication of the study for marketing professionals who are using nonmetric multidimensional scaling to aid in the understanding and prediction of consumer behavior.

THE HYPOTHESES

Product Class Experience and Dimensionality

While it was expected that such a respondent characteristic as education would have some influence on the number of dimensions used in making brand comparisons, it was felt that the major influencing role would be played by the respondent's product class experience. It can be argued that a consumer's product class experience has two major components: first, the experience a consumer obtains by exposure to product advertising and brand displays as he goes through the search and purchase decision steps in the buying process (shopping experience), and second, the experience the owner of a brand acquires as he uses a brand in a product class (usage experience). But what effect will the amounts of these two components of product experience have on the dimensionality of a respondent's perceptual map? We would argue that for inexpensive consumer durable products at least, previous shopping experience tends to increase the number of dimensions used in making interbrand comparisons, while previous usage experience tends to reduce the number of dimensions. In fact, for this class of products, we will be willing to make a more specific set of hypotheses based on the two components of product class experience. These hypotheses will concern three sets of consumers--those who have never owned a brand in the product class, and two subsets of product class owners: those who bought the item themselves and those who received it as a gift.

Our hypotheses about the relative numbers of dimensions employed by each of these groups are based on the supposition that product usage enables a consumer to discriminate among a product class' different attributes according to their saliences. The net effect of this process is that, with increasing usage experience, a reduction occurs in the number of dimensions used in making interbrand comparisons. For example, a person who has never owned an electric blender might be able to find four or five attributes that differ from one brand to another when he first compares brands. However, after using a blender for a few months, he would realize that only one or two of these product class attributes really matter to him. For this reason, we will hypothesize that the people who use the most dimensions will be the ones who have never before owned an item in the product class.

Next, the mediating effect of shopping experience will be considered. The discussion here will focus on the two classes of owners of the product: those who bought the item themselves, and those who received it as a gift. The difference between the two groups lies in the fact that, although both classes have product usage experience, only the people who bought the item themselves have shopping experience. What difference will this experience make in the number of dimensions each person employs in making comparisons between apirs of brands?

Those who bought the item themselves have both types of product class experience, but they are generally not unbiased in the number of dimensions they use. That is, in shopping for and selecting a brand, they based their decisions on the different brands' relative standing on several dimensions--a number of dimensions that, according to our earlier supposition, is probably larger than the number that a large amount of product use experience would demonstrate to be important. Both the theory of cognitive dissonance (Festinger, 1957) and the theory of self-perception (Bem, 1970) would predict that such a respondent, having undergone this shopping experience, would subsequently use more dimensions in making interbrand comparisons than product use experience alone would suggest. In other words, the number of dimensions used in subsequent decisions would be biased upward towards the number of dimensions the respondent used when making the purchase decision. On the other hand, the respondent who has received the product as a gift has not undergone shopping experience, and he will have no reason to subconsciously bias upward the number of dimensions used. Therefore, it seems reasonable to predict that the respondents who bought the item themselves will use more dimensions than the ones who obtained the item as a gift. To summarize, the results we would expect to obtain for all three classes of respondents are presented in Table l.

In a number of consumer durable product classes, particularly inexpensive shopping goods, many of the features that differentiate the brands in the store turn out to have little practical value in actual usage of the product. It is this group of products for which the model developed here has the greatest applicability, and it would seem that the product class used in this study, electric blenders, clearly falls in this category. Thus we would hypothesize that the relationships represented in Table l would prevail.

Dimensionality and Reliability

It also seems reasonable to hypothesize that the more dimensions an individual uses in making similarity judgments between brands, the less reliable he will be in making those judgments. Used in this sense, reliability refers to how well the subject can replicate his first set of judgments at a second point in time. The hypothesis is based on the fact that, in general, the more dimensions an individual uses, the more likely it is that he will develop an integratively complex system for combining them (Schroder, Driver, and Streufert, 196-, p. 7). In this type of system, as opposed to an information processing system with a low integration index, an individual can use different methods for combining dimensions in making interbrand comparisons, rather than using one system for all situations. It seems plausible to expect, then, that the individuals using the least numbers of dimensions are the ones who provide the most reliable similarity judgments. Because these people have the simplest information processing systems, they are unlikely to alter their framework of reference in the course of providing the necessary paired similarity comparisons.

TABLE 1

HYPOTHESIZED EFFECT OF PRODUCT EXPERIENCE ON THE DIMENSIONALITY OF PERCEPTUAL JUDGMENTS FOR DIFFERING AMOUNTS OF PRODUCT EXPERIENCE

THE DATA

The data used to test the hypotheses developed above were gathered as part of a project to test a model of consumer choice behavior. The subjects in the experiment were a paid sample of 188 women recruited by a market research recruiting firm. Although the women were not selected randomly, they did represent a wide variety of socioeconomic backgrounds. The stimuli in the project were twelve brands of electric blenders, a broad cross-section of the blenders available in retail outlets on the San Francisco Peninsula in March, 1972. The blenders were displayed in the laboratory in a manner similar to the wa they would be in the store, and the subjects had an opportunity to examine them thoroughly during the session. In the scenario which was used throughout the data gathering steps of each session, the subjects were told to imagine that they were buying a blender for their own use. If they already owned a blender they were told to imagine that it was broken and could not be economically repaired, and that they were therefore looking for a new one. To increase the realism of the situation, each subject was informed that several of the respondents would win one of the blenders and that the brand each of the winners would get depended on her choice behavior in the study.

Among the data provided by 74 subjects in one experimental condition [This subgroup of respondents differed from the other experimental conditions by not being exposed to a new brand priced at one of three different levels. For a complete description of the data collection procedure, see Ryans (1973).] were ratings of the overall similarity of all possible pairs of twelve blenders. The ratings were made on a nine-point rating scale anchored at "completely different" and "almost identical". In addition each subject also repeated the pairwise rating for four of the pairs of blenders (the same pairs were repeated in all questionnaires). The subjects were not informed that four pairs appeared twice in the list of seventy pairs of blenders, and none of them reported noticing the repetition. Seventy of the 74 subjects in this experimental condition provided sufficient data to be included in the analysis.

THE METHODOLOGY

In order to determine the number of dimensions each of the subjects was using in making her similarity judgments the individual similarity data matrices were first submitted to M-D-SCAL-V (Kruskal 1964a, 1964b), for scaling in five through one dimensions using Stess Formula One. The five stress values for each individual were then submitted to the M-SPACE program (Spence and Graef, 1973). M-SPACE is a program which determines the underlying dimensionality of an empirically obtained set of similarity data. The program is based on comparing the obtained set of stress values with those obtained in an extensive Monte Carlo simulation (Spence, 1970) in which the number of stimuli, the true underlying dimensionality, and the error level in the data varied. [Similar simulation studies have been performed by others (Young, 1970; Wagenaar and Padmos, 1971; Sherman. 1973: Isaac and Poor. 1974).] An index of fit is computed by taking the root mean square deviation of the fitted from the obtained stress values. The dimensionality and the error level at which the fit index is minimized is then determined and this is considered to be the dimensionality and error level that best characterize the data.

Since the data for each individual consisted of sixty-six paired comparisons on a nine-point similarity scale, there were obviously many tied ratings. The results reported in the next section are those based on using the primary approach to handling ties in M-S-SCAL-V, since it appears to be more commonly used in marketing applications. [In the primary approach no restrictions are placed on the fitted monotone regression values to a group of equal data values. In the secondary approach the fitted regression values are required to be equal when the original input data values are equal.]

Multiple regression was used to determine the relationship between the dimensionality obtained and such respondent characteristics as education and product class experience. Product class experience was represented by the square root of the number of times the respondent had used a blender in the previous month, and dummy variables were used to represent non-owners (n=6), and owners who received it as a gift (n=34). [The square root transformation was used since it was felt that the marginal effect of each additional use would decline as the total usage increased.]

A multiple regression analysis was also conducted to determine the relationship between dimensionality and reliability, while controlling on such variables as those representing product experience. The reliability with which a respondent was able to make the similarity judgments was determined by computing the Spearman rank correlation coefficient between the judgments for the four repeated pairs of blenders the first and second time they were made.

Ideally in both of the above sets of analyses we would like to have controlled on blender purchase and usage experience prior to obtaining the most recent blender. Unfortunately these data were not available. Since such experience would likely have occurred several years before this experiment, we do not think it would have a significant impact on the results reported here.

THE RESULTS

The aggregated results of the analysis to determine the dimensionality each subject was using in making her similarity judgments are reported in Table 2.

TABLE 2

THE NUMBER OF SUBJECTS USING DIFFERENT DIMENSIONALITIES IN MAKING THEIR SIMILARITY JUDGMENTS

The obtained dimensionality was then used as the dependent variable in the multiple regression analysis. Two models were used in the regression analysis, the first containing all the variables that it was hypothesized would influence the dependent variable, and the second containing only those that appeared to be most important. These results are reported in Table 3. The usage rate of the product class and the education of the respondent appeared to have no significant relationship with the number of dimensions that the respondent used, so these two variables were omitted in the second model. The product experience dummy variables both had the signs hypothesized, but only the variable representing owners who had received it as a gift was significant (p < .05, one-tailed test). That is, subjects who didn't own a blender tended to use about one-half of a dimension more than subjects who owned blenders which they had bought themselves. [This situation where subjects owned a blender and had bought it themselves is, of course, represented in the model by both dummy variables being zero.] Subjects who owned a blender received as a gift used the fewest dimensions, approximately one-half of a dimension less than those in the other owner category.

The results of the regression analysis examining the factors influencing the reliability of respondent similarity judgments are presented in Table 4. The first model did not include dimensionality as an independent variable. The results of this regression indicated that gift owners were significantly more reliable than the other classes of respondents in making their similarity judgments. This is to be expected given the previous results which suggested they used fewer dimensions. The hypothesized intervening variable dimensionality was introduced on models (2) and (3). In (2), dimensionality has, as hypothesized, a negative coefficient (p < .01, one-tailed test), indicating that as the number of dimensions a subject uses rises, his reliability tends to decline. It is interesting to note from ('! that even after dimensionality is entered into the regression model, the gift owner dummy variable continues to have a significant positive coefficient. In (3), a simple regression of reliability on dimensionality, it is also clearly indicated that for each additional dimension a subject uses, his reliability tends to be .17 lower.

Before turning to the implications of this study, it is appropriate to comment briefly on the relatively low R2 values reported in Tables 3 and 4. As Morrisson (1972) has pointed out, when one has a discrete dependent variable the R2 will be lowered somewhat, the amount depending on the true underlying distribution of the dependent variable and the number of discrete values the dependent variable is allowed to take. Therefore, the fact that dimensionality is a discrete variable with only four categories provides at least a partial explanation for the low R2 values obtained in Table 5.

The estimate of the reliability with which each subject is able to make similarity judgments is unlikely to be a very accurate one based as it is on only four repeated judgments. Ideally one would have liked to have a reliability measure based on a larger sample of repeated judgments. Thus there is likely to be a significant amount of error in the dependent variable in the regressions involving reliability. The effect of having this noise present in the observed reliability values is to reduce the observed R2, and this is probably another factor contributing to the relatively low R2 values observed in Table 4.

THE IMPLICATIONS

The results reported in the last section provide reasonable support for the hypotheses developed earlier. The effects of product class shopping and use experience on the number of dimensions that subjects used in comparing brands were in the hypothesized direction--blender owners who had received the brand as a gift tended to use about one less dimension than the respondents who did not own a blender. The analysis also suggested that the marginal impact of a subject using an additional dimension was a reduction of about .15 in her reliability (on a scale ranging from -1 to +1). Now that the hypotheses developed earlier do have some empirical support in one product class, we might tentatively suggest some practical implications for marketing managers and researchers using nonmetric multidimensional scaling.

Perhaps one of the most significant implications of the finding that product experience influences the number of dimensions used by a subject in making interbrand comparisons is that product experience may provide a useful basis for aggregation. That is, if groups with varying amounts of product class experience do use different numbers of dimensions then it is appropriate to first split the sample on this variable before submitting the data to an aggregate non-metric multidimensional scaling analysis, or to a "points of view" analysis. [A "points of view" analysis may be conducted by a Q-type principal component analysis of the type suggested by Tucker and Messick (1963) or by a clustering program such as Johnson's (1967) HICLUS program.] Segmenting the sample first on product experience has the practical advantage that it is a meaningful concept to marketing managers and it provides readily identifiable segments for developing specific communications plans. If the dimensions used by the various product experience segments are different or have markedly different saliences this may suggest that the evaluation criteria are quite different in the different segments. This again is useful information for managers charged with the task of developing an appropriate communications package. Furthermore, as a product class progresses through the product life cycle, the market changes from one where all respondents have little product class experience to a replacement market where practically all purchasers have at least some product class shopping or use experience. The company that fails to recognize this may be spending its communications budget very ineffectively.

TABLE 3

REGRESSION ANALYSIS OF VARIABLES INFLUENCING THE DIMENSIONALITY USED

TABLE 4

REGRESSION ANALYSIS OF VARIABLES INFLUENCING THE RELIABILITY OF A RESPONDENT'S JUDGMENTS

It would seem that the results of this study tend to confirm what some marketing managers have long suspected--that different comparison mechanisms are likely to be employed by shoppers who differ in their amount of experience with a product class. First-time buyers of an inexpensive durable good are likely to be influenced by several attributes of the products. On the other hand, experienced users buying the item as a replacement, are more likely to have settled on a relatively few salient attributes that they desire. These findings have some important implications for a marketing manager's product design and promotional strategies.

The fact that the number of dimensions used by a respondent has a significant effect on the reliability of a subject's similarity judgments also has important implications for designers of studies using nonmetric multidimensional scaling. Clearly if this result is generalizable to other product classes it is an important factor that must be considered in determining the sample size for a study where aggregation of the data is to occur. For example, within a segment of respondents who share homogeneous perceptions of the product class, the level of error in the aggregate data (e.g., the means of each paired comparison judgment across the sample) is a function of the sample size and the reliability of the individual subjects. If the subjects in a segment are less reliable, more subjects must be aggregated to obtain a given level of precision in the aggregate estimate. In general, an improved knowledge of the factors influencing reliability should result in better sample designs by marketing researchers in nonmetric multidimensional scaling studies.

To this point we have said little about the generalizability of these results to other product classes. It is probably reasonable to hypothesize that similar relationships between product experience, dimensionality and reliability occur in similar electric appliance or inexpensive durable product classes. The relationship between dimensionality and reliability should be even more generalizable; it probably holds for most, if not all, marketing situations where nonmetric multidimensional scaling might be applied.

Finally we would suggest that in any marketing research study using nonmetric multidimensional scaling it would be wise to conduct an analysis similar to the one described here either on the pretest data or on data from a small sample of subjects from the study. Such a study of product experience, dimensionality, and reliability, we contend would result in a better and more managerially useful analysis of the nonmetric multidimensional scaling data.

[The results of this study also have practical implications for the attitude models outside the multidimensional scaling framework (for example, the brand preference models of Bass and Talarzyk (1972) and Bass, Pessemier, and Lehmann (1973)),because the) indicate that in making cognitive comparisons at least, shoppers with no previous product experience are likely to use more dimensions than previous owners. Measuring more dimensions than previous owners really use might lead to noise in the data.]

REFERENCES

Bass, F. M., Pessemier, E. A., & Lehmann, D. R. An experimental study of the relationships between attitudes, brand preference, and choice. Behavioral Science, 1979, 18.

Bass, F. M., & Talarzyk, W. W. An attitude model for the study of brand preference. Journal of Marketing Research , 1972. 9, 93-96.

Bem, D. Beliefs, attitudes and human affairs. Belmont, California: Wadsworth Publishing Company, 1970.

Festinger, Leon. A Theory of cognitive dissonance. Evanston, Illinois: Row, Peterson, 1957.

Isaac, Paud D., & Poor, David. On the determination of appropriate dimensionality in data with error. Psychometrika, 1974, 39, 91-110.

Johnson, S. C. Hierarchical clustering schemes. Psychometrika, 1967, 32, 241-254.

Kruskal, J. B. Multidimensional scaling by optimizing goodness of fit to a metric hypothesis. Psychometrika, 1964. 29, 1-27. (a)

Kruskal, J. B. Nonmetric multidimensional scaling: A numerical method. Psychometrika, 1964. 29, 115-129. (b)

Morrisson, D. G. Upper bounds for correlations between binary outcomes and probabilistic predictions. Journal of the American Statistical Association, 1972, 67, 68-70.

Ryans, A. B. Estimating consumer preferences for a new durable brand at a giver price in an established product class--The development of a model and an experimental test. Unpublished doctoral dissertation, Stanford University, 1973.

Schroder, Harold M., Driver, Michael J., and Streufert, Siegfried. Human information processing. New York: Holt, Rinehart, and Winston, 1967.

Sherman, C. R. Nonmetric multidimensional scaling: A Monte Carlo study of the basic parameters. Psychometrika, 1,¦72, 37, 323-355.

Spence, Ian. Multidimensional scaling: An empirical and theoretical investigation. Unpublished doctoral dissertation, University of Toronto, 1970.

Spence, Ian. An aid to the estimation of dimensionality in nonmetric multidimensional scaling. Unpublished manuscript, The University of Western Ontario, 1971.

Spence, Ian, and Graef, Jed. The determination of the underlying dimensionality of an empirically obtained matrix of proximities. Multivariate Behavioral Research, 1974, 9, 331-341.

Tucker, L. R., and Messick, S. An individual differences model for multidimensional scaling. Psychometrika, 1963, 28, 333-367.

Wagenaar, W. A., and Padmos, P. Quantitative interpretation of stress in Kruskal's multidimensional scaling technique. British Journal of Mathematical and Statistical Psychology, 1971, 24, 101-110.

Wilkie, William L., and Pessemier, Edgar A. Issues in marketing's use of multiattribute attitude models. Journal of Marketing Research, 10, 428-441.

Wilkie, William L., and Weinrich, Rolf P. Effects of the number and type of attributes included in an attitude model. In M. Venkatesan (Ed.), Proceedings of the 1972 Annual Conference of the Association of Consumer Research pp. 325-340.

Young, F. W. Nonmetric multidimensional scaling: Recovery of metric information. Psychometrika, 1970, 35, 455-473.

----------------------------------------