A Comparison of Two Methods For Determining Optimum Levels of Product Characteristics

ABSTRACT - Optimum levels of intensity of physical attributes of food products can be determined in many ways. This paper compares results from two approaches: respondent ratings of an "ideal" product, and levels derived indirectly from cross-tabulations of beliefs ratings vs. overall evaluations. Results show that derived levels are higher than direct levels for most product characteristics.

Citation:

James H. Myers and Richard F. Chay (1979) ,"A Comparison of Two Methods For Determining Optimum Levels of Product Characteristics", in NA - Advances in Consumer Research Volume 06, eds. William L. Wilkie, Ann Abor, MI : Association for Consumer Research, Pages: 259-262.

Advances in Consumer Research Volume 6, 1979      Pages 259-262

A COMPARISON OF TWO METHODS FOR DETERMINING OPTIMUM LEVELS OF PRODUCT CHARACTERISTICS

James H. Myers, University of Southern California

Richard F. Chay, American Cyanamid, Inc.

ABSTRACT -

Optimum levels of intensity of physical attributes of food products can be determined in many ways. This paper compares results from two approaches: respondent ratings of an "ideal" product, and levels derived indirectly from cross-tabulations of beliefs ratings vs. overall evaluations. Results show that derived levels are higher than direct levels for most product characteristics.

INTRODUCTION

Product/service evaluation research is usually conducted for one or more of the following objectives: (1) diagnosis of the relative strengths and weaknesses of each product (brand) for the further objective of, (2) repositioning the firm's own offering(s), (3) redesigning existing products or brands, and (4) identifying opportunities for new products the firm might offer. Common to all of these objectives is the need to determine the particular combination of product characteristics, and the optimum levels of each characteristic, desired by the greatest number of consumers: the "ideal" product.

Determination of optimum levels for each characteristic is dependent upon the nature of the characteristics involved. Some are measured on a scale with discreet categories, others on a continuous scale. Some involve trade-offs (with price or with other features), other do not. Some involve subjective evaluations, others are objectively determined. No single research technique is available at the present time that will provide a solution for all contexts or conditions.

This study is concerned with determining optimum levels of product characteristics for an ethnic food product used for dinner occasions. The methodology required would also be applicable for most food, beverage, cosmetic, and household cleaning products; all of these types of products are composed primarily of attributes that are measured on continuous scales, are subjective, and do not involve trade-off considerations of any meaningful degrees (e.g., thickness, sweetness, saltiness, color saturation, strength of aroma, abrasiveness, carbonation).Such characteristics almost always require optimum rather than maximum levels, since consumers do not want "too much" of them. In contrast, people usually want all of the "benefits" they can get (e.g., durability, friendliness, convenience), within the normal range of human experience.

The problem is simply to determine, on a preliminary basis, optimum levels for the two to four most important product characteristics so that laboratory personnel can develop products at these levels. The matter of interaction effects is dealt with at a later stage of development, when several formulations are made up consisting of different levels of the most important characteristics that are at or near the optimum levels determined earlier.

Optimum Level Methodology

Researchers have approached the problem of determining optimum product characteristic levels in many ways:

1. Asking respondents to describe the characteristics of an ideal product using rating scales. The problems here are well known: respondents often do not know how important some features are (e.g., styling, packaging), they may be unwilling to divulge this information, they may be unable to describe an ideal product using rating scales (how would people have described a Coca Cola before this product existed?), and their ideal product ratings tend to be very similar to those for their "most preferred" brand or product (Bass, Pessemier, 1971; Carmone, 1970; Klahr, 1970; Johnson, 1971; Neidell, 1969; Wilkie, 1973).

2. Using experimental research to obtain reactions to varying combinations of product characteristics. While this is usually the best approach, it is often not feasible or even possible except in the case of food or beverage products whose formulations can be changed relatively easily. However, these changes are time consuming and expensive and they require prior information as to which 2-3 attributes are the crucial ones (Banks, 1965).

3. Using MDS and related technologies, comparing existing products directly against an ideal product, to derive ratio-scaled distance measures from which the characteristics of an ideal product can be inferred. However, the problem of clearly identifying the axes of the resulting positioning map is well known, and there is also the problem of translating the spatial location of an ideal product into measurable physical characteristics meaningful to R & D personnel (Green, 1975; Carmone, 1970). This latter problem is common to several other approaches, also.

4. Using the most preferred (for an individual) or the best selling (for the aggregate) product/ brand as an estimate of the ideal product. Measurable characteristics of this product can then be used as a reference point by laboratory personnel. Of course, the best existing product of a given type might or might not be similar to an ideal product; this would not be known unless additional research were done to determine both distance and direction of the "best" product to the ideal product. (Johnson, 1971).

5. Using optimization models (e.g., LINMAP) to estimate ideal points. This is more useful when the attribute scales and anchor statements used lead to interior (i.e., on the scale) ideal points rather than exterior (i.e., beyond the range of the scale, usually at the top or highest degree--of, say, durability) (Carroll, Chang 1965; Shocker, 1974; Zufryden, 1976). Also, these approaches often require more brands/products to rate than attributes on which to rate them, just the opposite of the usual situation in product development work.

6. Inferring optimal amounts of each characteristic from cross-tabulations of characteristic ratings vs. some meaningful dependent variable (e.g., overall evaluation, usage frequency). The problem here is that rating scale values (averages) cannot be converted directly into measurable physical characteristics, but they can be inferred from levels of these characteristics in other products rated. Also, neither this approach nor some of the others above indicate the first or second order interactions that often signal optimum combinations of product characteristics.

In view of the above shortcomings, it is obvious that there is no ideal method of determining the ideal product!

The present study involves a comparison of two of the above approaches; direct respondent ratings (#1 above) vs. derived estimates generated from cross-tabulations of each product characteristic against overall product evaluations (#6 above). The company involved was a large food processor that was interested in entering an established product category with a new product formulation.

METHOD

A total of 621 women who had used a brand in the product category in the past three months were recruited by telephone and brought to a central location facility. These women were members or friends of members of social and religious organizations in two cities: one large mid-western city and another smaller city in the southwest United States. It should be noted that women recruited from social and religious organizations are generally less representative than when conventional sampling methods are used.

Upon entering the central location facility, the women were asked to complete a background questionnaire to answer additional questions such as:

Frequency of category usage.

Brand used in past three months/brand used most often.

Satisfaction with usual brand.

Demographics.

After completing the background questionnaire, the women were asked to taste four formulations of the food product: two popular national brands, the company's own new formulation, and a regional brand relatively inferior to the other three brands tested.

Each woman was instructed to taste each product, drinking some cold water before and after the tastings. All of the women rated the four products on each of the eleven product characteristics plus an overall evaluation of the product. Pairs of words/phrases served as anchor points for a six-box scale. The characteristics selected for this taste test were developed jointly by marketing research, technical research, and marketing management departments, using focus group discussions and usage and attitude studies as input. All characteristics rated were descriptive rather than evaluative; the latter (e.g., good flavor)were eliminated since they would have had little or no diagnostic value for laboratory personnel. Examples of characteristics rated include thickness, chunkiness, saltiness, color saturation, and spiciness.

To maintain confidentiality, the actual product characteristic descriptors will not be used here, but will be designated by letters A-K. All polar descriptive words or phrases were of the form A, not A, B, not B; for example, extremely thick, not thick. To reduce the order bias associated with sequential evaluations, a strict rotation of all products and characteristics was maintained. Overall product evaluations were obtained on a six-point scale ranging from excellent to poor. After tasting and rating the four products, the women were asked to describe their ideal products using the same attribute scales. Averages of these ratings were used as coordinates for the ideal product in attribute space.

Direct Ideal Points

A mean value was calculated for ideal ratings on each of the twelve product characteristics. Table 1 shows frequency distributions for each of these characteristics for all respondents combined. This exhibit can be interpreted as follows: If Characteristic A were thickness (i.e., extremely thick = 1, not thick = 6), the ideal product would be somewhat more than midway between extremely thick and not thick (mean = 3.7). These mean ratings show optimum levels for each characteristic as developed directly from respondent ratings.

TABLE 1

DISTRIBUTION OF IDEAL RATINGS FOR PRODUCT CHARACTERISTICS: ALL RESPONDENTS COMBINED

Derived Ideal Points

Optimum levels for each characteristic were developed indirectly from cross-tabulations of characteristic ratings vs. overall evaluations, pooled across all products and respondents. An example is shown in Table 2 for Characteristic K (e.g., saltiness). Note how average overall evaluation ratings increase for the first four intervals on the scale, level off, and then decline. Thus, the optimum point for this attribute would be somewhere within the scale values 4 and 5. Note that Table 2 is simply a bivariate scatter plot for purposes of inspecting the form of the relationship between a dependent and an independent variable. In this case, the relationship is non-linear, with a correlation (eta coefficient) of .36.

TABLE 2

CROSS TABULATION OF RATINGS ON CHARACTERISTIC K VS OVERALL EVALUATION

The matter of how much of Characteristic K is represented by scale values 4 and 5 cannot be determined from cross-tabulation. It can, however, be inferred from mean ratings of each test products on the same scale.

Assuming Characteristic K can be measured by suitable laboratory instruments, the desired amount of this characteristic could be determined by interpolating from the amounts in each of the products rated.

Pooling ratings across all products is necessary in this type of analysis to provide a range of levels for each of the characteristics rated. A single product, or the same level of a given characteristic for all products, provides no opportunity for the variations that are essential for locating points where overall evaluation ratings are highest (optimum). Whenever highest overall ratings are found at either extreme of the scale, it indicates that the range is not sufficient to locate the optimum point, beyond which there is too much or too little of the particular characteristic. This can be remedied either by increasing or decreasing the amount of that characteristic for one or more test products (if this is possible) or by changing the descriptors at the ends of the rating scale into more extreme statements (e.g., very salty to extremely salty). The latter approach avoids truncation and allows respondents who do not like the saltiest product to so indicate through lower overall evaluations of this product, without being grouped with people who do not think this product is too salty.

Segmentation

Early in the analysis it was decided to see if meaningful segments could be found, based upon overall product preferences. Since the market for this food product was dominated by two somewhat dissimilar brands, respondents were separated into three groupings based upon their overall evaluation ratings for each of these brands: (1) prefer Brand A over B, (2) prefer Brand B over A, (3) prefer both equally (i.e., rate each at the same scale interval).

This procedure produced 269 respondents who preferred Brand A by one or more scale values on the 6-point overall evaluation scale, 266 who preferred Brand B, and 86 who were neutral. Each of the first two groups was considered to be a preference segment, and each was analyzed separately. Results are shown for each of these segments separately. To make certain that the two segments identified were not simply statistical artifacts, a significance test (t) was done of the differences in overall evaluation ratings of each of the two dominant brands between the two preference segments. Average ratings were found to be different at well beyond the .001 level of significance, suggesting that the segments were indeed different in their preferences for this ethnic food product.

Results

Table 3 shows both direct and derived ideal scale values for each of the preference segments. Table 4 shows the distribution of difference scores between the two types of ideal points.

TABLE 3

DIRECT & DERIVED IDEAL POINTS FOR TWO PREFERENCE SEGMENTS

TABLE 4

DISTRIBUTIONS OF DIFFERENCE SCORES BETWEEN DIRECT AND DERIVED IDEAL SCALE VALUES

Several points of interest emerge from these tables:

1. Twelve of the twenty-two comparisons of direct vs. derived ideal points differ by more than one interval on the six-point scale; three differ by more than two intervals.

2. Respondents consistently understated their ideal points on the direct basis, as compared to derived. In only four of twenty-two cases did respondents indicate an ideal point that was higher than the one from cross-tabulations; the remainder were lower.

3. The amount of this understatement of ideal points is itself understated, based on the comparisons shown in Table 3. This table shows a "6" for derived ideal points whenever the dependent variable rating (i.e., overall evaluation) increased monotonically for all intervals of the independent variable. This means, though, that the "topping out" point for such a characteristic might well be at some value one, two, or even more intervals beyond the six intervals on the rating scale for that characteristic.

4. The only consistent pattern between the two preferences segments was in the case of the three characteristics which related to food color; both groups showed about 2.5 scale units difference of direct vs. derived ideal points for Characteristic B, about 1.2 scale units difference for Characteristic J, and about 0.8 scale units for Characteristic E. Other than for color, there seemed to be no really consistent pattern of differences between the two segments. It is not clear why color was so much more consistent than the other features; perhaps there is more agreement for visual cues than for gustatory (taste).

In summary, it seems reasonable to conclude that there was not really good agreement between direct and derived ideal points. The former tended to be understated in relation to the latter, raising the question as to which of the two is the more accurate. Unfortunately, there is no answer to this within the data available for this study; the best answer would require experimentation using several products which varied systematically in the levels of all attributes rated. Such an approach was well beyond the resources available for this study.

At the very least, this study suggests that future research should not rely too heavily on subjective ideal point ratings from respondents. These should be supplemented by cross-tabulations of product characteristics vs. overall evaluation ratings, to provide at least another estimate of the optimum level of each characteristic. The latter approach has the advantages of objectivity plus reference points on each attribute based on average ratings for each of the products on each attribute. Optimum levels for each attribute can then be determined by interpolation or extrapolation from measurable levels on the products being tested.

REFERENCES

Seymour Banks, Experimentation in Marketing (McGraw Hill, 1965).

F. M. Bass, E. A. Pessemier, and D. R. Lehmann, "An Experimental Study of Relationships Between Attitudes, Brand Preference and Choice," Institute Paper No. 307, Lafayette, Ind., Krannert Graduate School of Industrial Administration, Purdue University, 1971.

J. D. Carroll and J. J. Chang, "Relating Preference Data to Multidimensional Scaling Solutions via a Generalization of Coombs' Unfolding Model," Bell Telephone Laboratories, Murray Hill, New Jersey.

J. O. Eastlack, "Consumer Flavor Preference Factors in Food Product Design," Journal of Marketing Research, 1 (February, 1964), 38-42.

Paul E. Green, "MDS & Related Technologies: Where Do We Go From Here?" Journal of Marketing, 39 (January, 1975), 24-31.

Paul E. Green and F. J. Carmone, Multidimensional Scaling and Related Techniques in Marketing Analysis (Allyn and Bacon, Boston, 1970).

D. Klahr, "A Study of Consumers' Cognitive Structure for Cigarette Brands," Journal of Business, 43 (April, 1970), 190-204.

Richard J. Johnson, "Market Segmentation: A Strategic Management Tool," Journal of Marketing Research, 8 (February, 1971), 13-19.

L. A. Neidell, "Physician Perception and Evaluation of Selected Ethical Drugs: An Application of Non-metric Multidimensional Scaling," dissertation, University of Pennsylvania, 1969.

Allan D. Shocker and V. Srinivasan, "A Consumer-Based Methodology for the Identification of New Product Ideas," Management Science, 20 (February, 1974) 921-937.

William L. Wilkie and Edgar A. Pessemier, "Issues in Marketing's Use of Multi-Attribute Attitude Models," Journal of Marketing Research, 10 (November, 1973), 428-441.

Fred S. Zufryden, "A Zero-One Integer Programming Approach to Market Segmentation & Product Positioning," GSBA Working Paper, School of Business Administration, University of Southern California, 1976.

----------------------------------------