The Compensatory Dimension in Subjective Evaluation Processes:&Nbsp; a Multimethod Validation

Ruby Roy Dholakia, Kansas State University
ABSTRACT - One of the characteristics of evaluation processes of multi-attributed alternatives is that of compensatoriness which defines the nature and degree of trade-offs between attributes. An empirical investigation is reported in this study that utilizes multiple methods to analyze this dimension. The objective of the paper is to represent the compensatory dimension in terms of a single quantitative measure and find qualitative support for the reported findings. Judgments of three decision makers are analyzed using these methods and the findings are then used to select representation models of the evaluation processes.
[ to cite ]:
Ruby Roy Dholakia (1981) ,"The Compensatory Dimension in Subjective Evaluation Processes:&Nbsp; a Multimethod Validation", in NA - Advances in Consumer Research Volume 08, eds. Kent B. Monroe, Ann Abor, MI : Association for Consumer Research, Pages: 362-366.

Advances in Consumer Research Volume 8, 1981      Pages 362-366


Ruby Roy Dholakia, Kansas State University


One of the characteristics of evaluation processes of multi-attributed alternatives is that of compensatoriness which defines the nature and degree of trade-offs between attributes. An empirical investigation is reported in this study that utilizes multiple methods to analyze this dimension. The objective of the paper is to represent the compensatory dimension in terms of a single quantitative measure and find qualitative support for the reported findings. Judgments of three decision makers are analyzed using these methods and the findings are then used to select representation models of the evaluation processes.


Early attempts to describe and analyze evaluation processes revolved around the nodes versus process controversy (Goldberg 1968, Wright 1975). While there is ample evidence to support differences in processes, both across individuals and situations, these differences are still viewed at the model level (Park 1976). Recent approaches have been attempting to isolate these differences at specific dimension level such as the influence of time horizon on process linearity (Wright and Weitz 1977).

When multiple attributes are used by individuals to evaluate alternatives, one component of the evaluation process is that of inter-attribute compensation. Two attributes are compensatory when a decrease in the value of one is balanced by an increase in the value of the second attribute. The amount required for balancing is likely to vary and will be determined by a variety of factors. Evaluation processes will differ, therefore, in terms of compensatoriness.

In traditional economic theory, the marginal rate of substitution between two goods was an index of compensation (Henderson and Quandt 1971). Recent formulations have shown the preferences for goods to be derived from the preferences for the goods' attributes (Lancaster 1966) and the marginal rate of substitution has been extended to relationships between attributes (Lancaster 1972). Raffia (1968) showed through isopreference curves that the rate of substitution between an unit of attribute X-and an unit of attribute Y can be constant or variable. A variable substitution rate depends on the specific attribute levels while a constant rate is independent of attribute values.

The compensatory dimension can, therefore, be represented as a continuum (degree) rather than as a dichotomy (compensatory). This implies that evaluation of multi-attributed objects rather than being categorically a compensatory or a non-compensatory process, can be characterized as a process with varying degrees of compensatoriness. Raffia's (1968) variable rate of substitution supports this view. In the utility model proposed by Keeney (1974), the different values that the multiplicative constant W can assume is based on the variable rate of substitution.

There are two important implications of viewing the compensatory dimension as a continuum. When various models are used to describe a process or predict its outcome, the current practice is to distinguish these models as being compensatory or non-compensatory. However, if the compensatory dimension is viewed as a continuum, then greater discriminations must be made among these models and their assumptions about the degree of compensatoriness.

Second, it allows conceptualization of the influence of various situational and individual variables on the evaluation process. For example, the intersection of a specific set of variables may be to make the evaluation process more or less compensatory. If a methodology can be adopted to quantify the degree of compensatoriness, then changes in magnitude can be hypothesized and measured.

Investigation of Compensatoriness

There is limited empirical evidence pertaining to the rate of compensation among attributes. Utility theorists have used goods rather than attributes and assumed the goods to be use-attributed. MacCrimmon and Toda (1969) used money and ballpoint pens in one study and money and pastries in the second. Similarly, Rousseau and Hart (1951) used goods rather than attributes when obtaining trade-offs between bacon and eggs.

There are some attempts by consumer behavior researchers to assess attribute trade-offs. Johnson (1974) reported a methodology which used paired comparisons to obtain trade-offs from which utilities of individual attributes as well as overall utility were derived. Fiedler (1972) used the methodology to obtain evaluation of condominiums and found it to be a good predictor of preferences. Hauser and Urban (1979) used an alternative methodology proposed by vonNeumann-Morgenstern and found considerable individual variation in trade-offs as well as inter-attribute differences. The focus of these papers has been to assess pair-wise trade-offs directly and to derive the overall utility of alternatives from the analysis of trade-offs.


A research study was designed to investigate the compensatory dimension of evaluation processes. Unlike earlier studies, the attempt vas not to obtain overall utility but to focus on the compensatory dimension per se. The objective was to represent the dimension in terms of a single quantitative measure that can be used as an indicator of the degree of compensatoriness in a process. Since the methodological issues were of primary concern, the empirical investigation was confined to a study of career opportunities by MBA candidates.


Students from a graduating MBA class were requested to participate in a decision making study. Since career placement loomed large in their lives at that point in time, job choices were used as the subject of investigation. This ensured cooperation and involvement from the participants.

Independent Variables.  A set of attributes relevant to job choice among the graduating class was identified from group responses. In a classroom setting, the students were asked to list the five most important attributes in job selection. Five attributes were identified by taking into account frequency of mention and the average importance rating assigned to each. The choice of these five attributes were validated among a second group of students. From this emerged a reduced set of three attributes which was considered most important by majority of the students.

These three attributes (initial salary, job location and responsibility) were used to develop alternative job opportunities. Twenty-seven jobs were generated from three levels of each attribute:

Salary: $14,000; $16,000; $20,000 annually;

Location: Northeast (NE), Midwest (MW), and Farwest (FW);

Responsibility: Low, moderate and high.

A second set of .18 (2X3X3) jobs was composed for validation purposes using a subset of the attribute values: moderate and high levels of responsibility, the same locations and $12,000; $15,500 and $18,000 salaries. Each job was described in terms of these attribute values on a 3x5 card.

Dependent Variables.  A series of measures were obtained for the job evaluation task:

a) Measure of liking for each job on an 11-point scale (0-10);

b) Measure of preference order through successive entries into most and least preferred categories;

c) Verbalization of thoughts underlying the sorting procedure as the sorting took place. (This was tape recorded and later transcribed).

The first set of 27 Jobs vas evaluated along each of these measures. The validation set of 18 jobs was evaluated only in terms of (a) above. The entire evaluation task took an average of about 80 minutes to complete.

Participants.  Twelve decision makers participated in the study. They were all graduating MBA students in a mid-western university, seeking permanent placement and had volunteered for the study. A token payment of five dollars was made to each participant.


Model Performance.  Since one of the objectives for identifying the compensatory dimension was to improve the selection of representational models, the first analysis performed was to predict the job preferences by the use of different models. Three different models - regression (reg), additive conjoint (con) and lexicographic (lex) -were used. The three models assume different degrees of compensatoriness: in this research they could only be ordered in terms of this assumption, with the lexicographic model being the most compensatory (rank 1) and the regression model being the least compensatory (rank 3).

Table 1 provides the predictive validity of the three models using two criteria - recovery of the rank ordering of the entire job set and the accuracy in predicting the first choice.



As we can see there are some interesting differences in the validity of these models as representations of the process. Using both criteria of predicting the rank order and the most preferred job, four categories of model performances may be observed. In the first category, all the three models do equally well in predicting the rank orders and first choices for some decision makers (DM 2, 5, 8, 10 and 11). For DM 7, the competition is between two models - the conjoint and lexicographic. This category represents the maximum ambiguity since the models perform similarly although making different assumptions about the degree of compensatoriness in the underlying process.

Category two, represented by DM 12, generates some amount of ambiguity because the three models differ in their predictive validity - the regression model recovers the rank order the best but fails to predict the first choice while the lexicographic model predicts the first choice successfully but performs the worst in recovering the rank order.

The third category appears to be unambiguous because it is possible to select a model based on both criteria. For DMs 1, 3, 6 and 9, the model that predicts the rank order the best is also able to predict the first choice. Finally, the fourth category (which only includes DM 4) represents a situation where none of the three models perform satisfactorily on both criteria. To allow detailed exploration of the problem, further analysis and discussion is limited to three decision makers (labeled A, B, C) conveniently chosen as representatives of the three categories. Category four has besn excluded because none of the models performed satisfactorily.

The Compensatory Dimension.  Given the observed differences in model performances and the ambiguity in selecting a representational model, the objective of further analysis was to directly use the compensatory dimension in evaluating alternative models. To achieve this objective it is necessary to represent each decision maker on this dimension and a composite index of compensatoriness was devised for this purpose. (Technical details are given in the Appendix).

The values of the composite index are reported in Table 2 for the three selected decision makers. Higher the value, the more noncompensatory the process since it implies larger units of an attribute being required to compensate for another attribute. A perfectly compensatory process would be represented by a zero value in terms of the composite index of compensatoriness.



According to Table 2, DM C exhibits the most compensatoriness in the job evaluation process while DM B is the least compensatory. Given this information, it would appear that the lexicographic model is a better representation of DM B's evaluation process but is not appropriate for DM C's process. On the other hand, the regression model which assumes a high degree of compensatoriness may be used to characterize DM C's process. By similar logic, the conjoint (additive) model, which allows for somewhat less compensatoriness in the process than the regression model, may be used to characterize DM A's process. Thus by using the compensatory dimension directly, it is possible to select representational models that are more appropriate for a decision maker based on both predictive criteria and the assumption about the level of compensatoriness in an evaluation process.


To validate the findings on the level of compensatoriness, one other analysis was performed. This was done to ensure that the values of the composite index was not an artifact of the computation process. Verbal protocols were analyzed to see if trade-offs were considered by the decision makers in evaluating the alternative jobs. The verbalizations took place while the first set of 27 jobs were being successively sorted into preference categories. They were tape recorded and later transcribed. Excerpts from the protocols are reported in Exhibit 1.



Analysis of the verbal protocols support the quantitative descriptions of the evaluation process in terms of compensatoriness. As we can see from the excerpts, DM A allows some amount of compensation; for instance, between a Midwest location and a low salary but not a lot, since low responsibility is not being balanced by a higher salary or an attractive location. On the other hand, DM B is quite strict about attributes desired in a job and is unwilling to like a job unless it meets the attribute preferences; even a $4,000 extra in initial salary cannot compensate for a less desirable Midwest location. Much greater balancing of attributes takes place in DM C's process, for instance, when a better salary is accepted even if the responsibility is not desirable.

Model Selection.  To the extent that the three models -regression, conjoint and lexicographic - can be ordered in terms of their assumption regarding process compensatoriness, these models may then be used to represent the process of the three job candidates. The lexicographic model which assumes the highest level of noncompensatoriness is therefore selected as the best representational model for DM B even though the other two models predict the job choice and rank order.

The predictive validity of the conjoint model is supported by the level of compensatoriness found in DM A's process. This level validates the conjoint (additive) model's assumption of a limited degree of compensatoriness. For DM C, the regression model is found to be the best representation because it assumes a high degree of compensatoriness which was found to characterize DM C's process.

Selection Validity.  The descriptive and predictive validity of each model selected to be representative of the three processes are reported in Table 3. The correlations may be compared to those of the conveniently accessible regression model.



There is an improvement in both descriptive and predictive validity by the use of a specific model even though the improvement may not be statistically significant. However, the justifications for selecting a model to represent the process are built on different grounds. A valuable illustration of the limitation of statistical considerations is made by DM B; despite high predictive validity of the regression model, it makes a totally inaccurate assumption about the compensatory dimension of the underlying subjective process.


It appears that focusing on the compensatory dimension of the evaluation process provides a specific criterion for discriminating and selecting among representational models. Despite the very high predictive validity of all the models, there is support for the noncompensatory assumption of the lexicographic model which makes it more valid for representing DM B. Similarly, the conflict created by the overall predictive validity of the regression model and the first choice accuracy of the lexicographic model for DM C's process could be resolved by an analysis of the compensatory dimension. The regression model is found to be a better representation of the underlying process because of the degree of compensatoriness. Superior predictive validity of the conjoint (additive) model for DM A is supported by the findings on process compensatoriness.

Even the use of multiple measures of predictive validity (overall order and the first choice) is seen to be insufficient for determining the nature of evaluation processes implied in the job preferences. Identifying and measuring the compensatory dimension of the evaluation process is found to contribute to its understanding and representation by models. The attempt to represent the magnitude of the compensatory dimension appears also to be fruitful.

There are substantial methodological differences in this paper as compared to other studies of process compensatoriness. Past studies that have focused on this dimension have used it to derive overall utility. While that objective has been successfully achieved, it often involves use of a methodology that is quite restrictive in its usefulness (Hauser and Urban 1979). It required a high degree of conceptual ability for the respondents to understand the procedure by which trade-offs were measured. In contrast, this study has focused on the compensatory dimension without requiring the decisionmakers to undertake any task that is difficult or not normal. Instead, it has adopted a reverse route -- moving from overall measures of evaluation to a specific measure of process compensatoriness.

The approach adopted in this paper for investigating subjective evaluation processes appears to have several other merits. By focusing on a specific dimension and by viewing the dimension on a continuum, it is possible to relate the effects of individual and situational variables on the evaluation process. Instead of just saying that these variables are likely to create differences, it is possible to say where the difference is likely to exist and the nature of the difference (i.e. more or less compensatory). One can hypothesize that increased involvement with a product or issue will reduce the compensatoriness of a process and can test the hypothesis. Such an approach is likely to be more productive in creating an understanding of subjective evaluation processes.

By restricting the study to three decisionmakers, the validity of the approach is open to question. However the rationale of the approach cannot be tested by numbers alone The objective of the approach was to understand the subjective evaluation process in terms of a single dimension -compensatoriness. It is therefore necessary to directly portray evaluation processes on this dimension and to select models as representations when they satisfied the assumption on this dimension and predictive criteria. The approach offers an alternative to the model vs. process controversy by selecting models based on an understanding of underlying process dimensions.

To fully benefit from this approach, it is necessary to explore another implication, viz to generate other dimensions of the evaluation process and to determine their relationships to the compensatory dimension. The compensatory dimension is only one component of the evaluation process.

While there is some literature on other dimensions (e.g. linearity, configurability, etc.) of the evaluation process, there is a lack of concepts and theories that link these together and create an integrative framework for the conceptualization and understanding of subjective evaluation processes. There exists a need for developing such a framework that can put past research in perspective and identify future research issues. This need is strengthened by recent findings that individual evaluation processes are likely to be constructive and not rule or model applications (Bettman and Zins 1977).

Even though the investigation reported in this paper deals with job choice, it focuses on the compensatory dimension which is likely to be of even greater relevance to study of traditional consumer goods. In the present context of shortages and inflation when significant changes are likely in consumer processes, the impact on the compensatory dimension requires major attention.


The compensatory dimension was represented by a sec of values computed from the attribute utility scores obtained from conjoint analysis. In order to reach the overall index of compensatoriness, the following calculations were performed.

1.  Compensation between pairs of attributes:

a.  Responsibility and salary,

b.  Location and salary,

c.  Location and responsibility.

The rate of compensation is obtained for each change in the level of a pair of attribute, e.g. the change in utility associated with an increase in salary from $14,000 to $16,000 and from $16,000 to $20,000 as compared to change in utility with an increase in responsibility from low to moderate and moderate to high. The ratio of this change U(16)-(14)/U(Mod)-U(Lo), is taken as a measure of attribute compensation. Similarly, for the other two pairs.

2.  Perfect rate of compensation is assumed to exist when the ratio equals one. An assumption here is that the unit of change in the attribute is same across different levels (e. g, moderate - low = high - moderate).For salary, an adjustment has been made to incorporate the different levels of change (20-16 = 2(16-14)) which leads to a perfect rate of 0.5

3.  The composite index of process compensatoriness is obtained by the following:

S|Xij - Xj|

where: Xij = compensation between a pair of attribute level

Xj = rate of perfect compenstion for the same pair.

For example, the composite index for DM A is found by incorporating the following values:

12.94 = *(3.92-0.5)+(1.88-1.0)+(5.27-0.5)+(1.14-1.0)+(0.61-1.0)+(-1.34-1.0) *

The detailed values are given below:




Bettman, J. R. and Zins, M. A. (1977), "Constructive Processes in Consumer Choice," Journal of Consumer Research, 4, 2, 75-85

Fielder, J. A. (1972), "Condominium Design and Pricing: A Case Study in Consumer Trade-Off Analysis," Proceedings, Association for Consumer Research, 279-93

Goldberg, L. R. (1968), "Simple Models or Simple Processes: Some Research on Clinical Judgment," American Psychologist, 23, 483-96.

Hauser, J. R. and Urban, G. L. (1979), "Assessment of Attribute Importances and Consumer Utility Functions: vonNeumann-Morgenstern Theory Applied to Consumer Behavior," Journal of Consumer Research, 5, 4, 251-62.

Henderson, J. M. and Quandt, R. E. (1971), Microeconomic Theory: A Mathematical Approach, Second Edition, New York: McGraw Hill.

Johnson, R. M. (1974), "Trade-off Analysis of Consumer Values," Journal of Marketing Research, 11, 121-27.

Keeney, R. L. (1974), "Multiplicative Utility Functions," Operations Research, 22, 1, 22-3.

Lancaster, K. J. (1966), "A New Approach in Consumer Theory," The Journal of Political Economy, 74, 2, 132-57.

Lancaster, K. J. (1972) Consumer Demand: A New Approach, New York: Columbia University Press.

MacCrimmon, K. R. and Toda, M. (1969), "The Experimental Determination of Indifference Curves," Review of Economic Studies, 36, 433-51.

Park, C. W. (1976), "The Effect of Individual and Situational-related Factors on Consumer Selection of Judgement Models," Journal of Marketing Research, 13, 144-51.

Raffia, H. (1968), Decision Analysis, Reading, Mass: Addison-Wesley.

Rousseau, S. N. and Hart, A. G. (1951), "Experimental Verification of a Composite Indifference Map," The Journal of Political Economy, 59, 288-318.

Wright, P. L. (1975), "Consumer Choice Strategies: Simplifying vs. Optimizing," Journal of Marketing Research, 11, 60-7.