Making Decisions With Incomplete Information: the First Complete Test of the Inference Model

ABSTRACT - The present study is the first test of the inference model to experimentally control interattribute correlations and the importance of the missing attribute; thereby directly testing causal relationships. All the predictions of the inference model were strongly supported; specifically a) available information has a greater influence on the judgments when it is positively correlated with missing information and a diminished influence when the correlation is negative, b) the correlation effects and the bias against missing information are greater when the missing information is more important, and c) framing effects are independent of the weight of the missing information.

Citation:

Richard D. Johnson (1989) ,"Making Decisions With Incomplete Information: the First Complete Test of the Inference Model", in NA - Advances in Consumer Research Volume 16, eds. Thomas K. Srull, Provo, UT : Association for Consumer Research, Pages: 522-528.

Advances in Consumer Research Volume 16, 1989      Pages 522-528

MAKING DECISIONS WITH INCOMPLETE INFORMATION: THE FIRST COMPLETE TEST OF THE INFERENCE MODEL

Richard D. Johnson, University of Alberta

ABSTRACT -

The present study is the first test of the inference model to experimentally control interattribute correlations and the importance of the missing attribute; thereby directly testing causal relationships. All the predictions of the inference model were strongly supported; specifically a) available information has a greater influence on the judgments when it is positively correlated with missing information and a diminished influence when the correlation is negative, b) the correlation effects and the bias against missing information are greater when the missing information is more important, and c) framing effects are independent of the weight of the missing information.

The question of when, and how, subjects make inferences about missing attributes is fundamentally important for both basic and applied research.

Inferences and Basic Research

Virtually all cognitive and economic models of decision making assume that decision makers use the information presented to them; or some subset of that information in the case of heuristics or satisficing approaches. Inference models raise the possibility that people may, in fact, employ inferences to process more information; a supers t of what is presented.

The demonstration that people elaborate on, rather than simplify, the information set requires a reevaluation not only of established theories, but also of methods of the study of judgment and decision making. For example, attempts to identify the process by which attribute information is combined often depend on manipulating the amount of information presented to the subject (Anderson, 1981). These methods are valid only if we can assume that the subject ignores, or is unaware of, attributes for which information is not presented on some of the trials (Johnson and Levin, 1985; Cohen, Miniard, and Dickson, 1979).

The inference models also show promise for increasing our understanding of information search (Meyer, 1982) and the phenomenon of overconfidence in one's judgments and decisions (Levin, Chapman, and Johnson, 1988).

Inferences and Marketing Strategy

Consumers' inferring of missing information signifies a number of important implications for marketing strategy and research; both in suggesting useful procedures and in identifying potential traps. A number of companies' marketing strategies appear to be designed to take advantage of inferences. For example a national brand of shampoo advertises "It's more expensive, but I'm worth it;" encouraging people to (perhaps incorrectly) infer high quality from the premium price. AT&T's advertising has taken a similar tactic, reminding consumers that "You get what you pay for."

At the same time, marketing managers must be aware of, and design marketing strategies to avoid, traps that might result when consumers infer missing information. For example, a local grocery store recently offered ground beef at 994 per pound. The advertising copy, package, and point of sale display provided no cues to the percentage of fat in the meat. Many consumers may have inferred that the low price signalled poor quality and passed up the bargain ground beef. The marketing strategy was improved when the offer was repeated with the words "regular quality" prominently displayed in advertisement, package, and display.

Inferences and Applied Marketing Research: An Empirical Example

The impact of inferences on marketing research can be illustrated by a study conducted in our laboratory (Johnson and Tan, 1987) concerning the effect of inferences on the results of conjoint analysis research.

Rationale. Respondents to conjoint analysis surveys judge hypothetical products which are created by the factorial combination of pre-specified attributes. The number of hypothetical products, and therefore the length and difficulty of the survey, are directly related to the number of attributes and attribute levels to be tested. Hence, the number of attributes is usually limited by a constraint on the number of -trials a respondent is expected to process. For this reason, researchers are often tempted to omit an attribute that is not relevant to the management decision. Such a strategy can have serious consequences for the respondents' decisions and, hence, for the attribute importance estimates derived from the conjoint analysis.

Consider, for example, a manufacturer who may offer televisions of varying screen sizes and prices. A survey may be conducted in order to identify the size and price combination with the greatest likelihood of success. If no information about warranty is available to respondents, they may infer it from the price (Johnson and Levin, 1985) or from the screen size. More expensive televisions, all things being equal, are less attractive; but if they are expected to have better warranties than less expensive televisions, then the negative effect of progressively worse prices may be mitigated by inferences about progressively better warranties. The effect of price will be smaller when respondents infer warranty, than when they do not.

Our research was motivated by two questions. First, do inferences matter in conjoint analysis research in which missing attributes are never presented? This approach differs from most published studies of the inference model which make the "missing" attribute salient by presenting it on some trials and omitting it on others; a procedure that may lead to experimental artifacts (Johnson, Levin, and Cressey, 1986; Lim, Olshavsky, and Kim, 1988). Second, if inferences affect applications of conjoint analysis, can their effects be easily and inexpensively avoided?

Method. Two different surveys were distributed to a convenience sample of fifty-seven students at the University of Alberta. Both conditions were identical, except for the instructions on the cover page. Subjects were asked to evaluate 36 hypothetical television descriptions, which were generated from a full factorial (22 X 32) of Price ($250, $450, or $650) X Screen Size (14, 20, or 26 inches) X Brand (RCA or Toshiba) X Remote Control (yes or no). In the control condition (N=26), subjects were given the standard instruction to "assume that everything else about the television sets is acceptable to you." Subjects in the experimental condition (N=31) were told explicitly that "All the TV sets feature a full warranty for 3 years. Please assume that everything else about the television sets is acceptable to you."

If subjects inferred the length of warranty from price in the standard instructions condition, then the effect of price on the evaluations will be reduced relative to the experimental condition in which the length of warranty was explicitly provided. The effect of screen size will be enhanced by inferences that larger televisions have longer warranties.

Results. The results are consistent with the prediction that subjects' judgments were influenced more by inferred warranty information in the control condition than in the experimental condition. Specifically, the observed effect of price on subjects' judgments was reduced when subjects were allowed to infer that inexpensive televisions may have poorer warranties (and vice versa). Furthermore, the effect of screen size was greater in the control condition; as predicted by the hypothesis that subjects inferred that larger televisions usually have better warranties. This is an important finding because it demonstrates simultaneously, that a) people can and do infer values of missing attributes even when those attributes are not made salient in a task; and b) the effect(s) of inferences on judgments can be substantially reduced or eliminated by a simple editorial change in the survey instructions.

The Inferring Missing Values Model

Johnson and Levin (1985) proposed and tested a model describing the pattern of inferences and their influence on the overall judgment of the alternative. This model incorporated both within-brand and between-brand attribute information. According to the Inferring of Missing Values model, subjects use available information to make inferences about the levels of the missing attribute. They proposed that the inference is linearly related to the known information. In the two attribute case, where al is presented and a2 is missing,

a₂' = ma₁ + k, (Equation 1)

where m represents the perceived relationship between the subjective value of attribute 1 and the subjective value of attribute 2, and k represents the mean level of the inference.

The relationship between the attributes is hypothesized to operate at a subjective level. For example, consider a grocery store that offers ground beef for $.99/lb, but does not reveal the percentage of fat in the meat. The Johnson and Levin model describes a process in which shoppers think "99 cents is a good price, so it must be pretty bad quality"; rather than "99 cents means it must be about 40 percent fat." Consequently, inferences are not subject to framing effects that are observed when the presented information is associated with a label, such as percentage of fat (or lean).

The general level of the inference, k, depends upon the expected mean or "normal" level of the missing attribute and on a bias that occurs if the subject assumes that the information has been withheld because it is unfavorable. The latter has been called an adjustment for uncertainty (eg., Meyer 1981) or a bias against missing information (Johnson, 1987f The expectations about the missing attribute are a function of experience. Consider, for example, a consumer who evaluates hypothetical pizza restaurants for which the price of a large cheese pizza varies around a mean of $10. Suppose, however, that the consumer perceives the price of comparable pizzas in established restaurants in his town to be distributed around a mean of $12. Inferences based on an expectation of the higher price should produce less favorable evaluations for missing information trials compared with full information trials. This result might be interpreted as a bias against missing information; although it results from a different process from one in which a subject discounts alternatives with missing information or infers that the information is unfavorable because it was withheld.

Another way to think about equation (1) is in terms of an intra-alternative component (m) which is based on information about other attributes of the same brand, and an- inter-alternative component (k) which depends on information about the same attribute across brands (see Zwick, 1988). Johnson (1987) demonstrated that these two components could be manipulated independently; the former was increased by a simple reminder of the correlation between attributes. and the latter by framing effects and product knowledge. This paper further explores these and other factors that influence inference processes.

The inference is integrated with the presented information to reach a final judgment. Assuming an additive integration rule (both adding or averaging lead to the same predictions), the information for the two attributes in the full information trials may be combined according to the following model (Anderson, 1981).

R = w₁a₁ + w₂a₂     (Equation 2)

The relative influence or weight of each attribute, ai, is represented by wi. When information for attribute 2 is missing, a subject may substitute the inference:

R = w₁a₁ + w₂(ma₁ + k)

   = (w₁ + mw₂)a₁ + w₂k    (Equation 3)

Equation (3) takes the form of a linear function where the slope of R as a function of attribute 1 depends on w₁, w₂, and m, and the intercept depends on w₂ and k. The specific predictions that follow from equation (3) are described in the next section.

PREDICTIONS

Effects of Intra-Alternative Information

The effect of the presented information on the ratings depends on the perceived relationship between presented and missing information. Recall that R = (w₁ + w₂m)a₁ + w₂k. Thus, the effect of the slope term, a₁, on R is w₁ + w₂m.- Compare this to the case where all the information is available, and the effect of a₁ on R is w₁. When information is missing, an inference increases the effect of a .presented attribute if m>0, and decreases it if m < 0. Further, the effect of the relationship interacts with the weight of the missing attribute. The dependent measure is the estimate of the effect on ratings (slope estimate) for either of the two attributes as a function of the amount of information presented.

Prediction 1a) Positive relationship: The inference is subjectively similar to the known information and enhances the effect of presented information.

Prediction 1b) Negative relationship: The inference is opposite to the known information, and reduces the observable effect of the presented information.

Prediction 2): The weight of the missing attribute interacts with the inference. The more important the missing information, the greater should be the effects that are predicted in la and lb.

Effects of Inter-alternative Information

Several processes have been proposed to affect the mean level of subjects' evaluations of completely and incompletely described alternatives, regardless of the values of the presented information. The first, somewhat misleadingly called a bias, is represented by k in the inference model. As described above it is a function of the expected mean or "normal" value of the missing information and of a negative reaction to the fact that information is withheld. According to the model, it interacts with the weight of the missing information. The second process, a framing effect, has been seen in trials when all the information is presented and not seen when information for a key attribute is missing. This finding has been interpreted as support for the hypothesis that inferences represent subjective valuations of the attribute level, and are not associated with the label that leads to framing of the same attribute in the completely described alternatives (Johnson, 1987; Levin, Johnson, Russo, and Deldin, 1985). The dependent measure is the mean of the evaluations, as a function of amount of information presented.

Prediction 3: The bias against missing information will reduce the attractiveness of the inference and lead to less favorable evaluations. It is as if subjects infer that "it must be bad news if it is being withheld".

Prediction 4: The more important the missing information, the greater the effect of the bias.

Prediction 5: A framing effect will appear only in fully described trials. Mean ratings for trials with missing information will be not differ across framing conditions.

Prediction 6: Since the frame of the missing attribute does not affect the mean level of the inference, k, then increasing the weight of the missing information will not increase (or make visible) a framing effect for trials with missing information.

Prediction 7: Since the frame of the presented information affects the perception of that attribute, and the perceived value of the presented information is, in turn, used as a basis for inferring the missing information, a positive relationship between the attributes (m > 0) should enhance the framing effect and a negative relationship should reduce it.

One might note that the model also appears to predict that the effect of the inter-attribute relationship on the frame will be greater when the missing information is relatively more important. However, this prediction holds only if the weights of the attributes are independent, as in a classic adding model. The averaging model predicts that an increase in W2 means a decrease in w1, negating the effect. In fact, the weight of the evidence supports the averaging model (Anderson and Lopes, 1974; Birnbaum, Wong, and Wong, 1976); so this interaction is not expected.

METHOD

Subjects were 160 introductory psychology students at the University of Alberta whose participation in this study was in partial fulfillment of a course requirement. Each subject rated 44 hypothetical candidates for a position selling a microcomputer accessory. The accessory, called the "Comptroller", was described as a new product about to be launched for the first time. Each subject was asked to take the role of a sales manager faced with the task of hiring a sales force to promote the new product.

The hypothetical job candidates were described as having been asked to take two examinations: one to test the candidates' knowledge of sales techniques, and the other to measure the candidates' knowledge of computers. Candidates' suitability for the job were judged on a twenty point scale, based on these two test scores. Some candidates were said to have been "unable" to take one of the tests, and subjects were asked to base the rating for those candidates on the available information, i.e. the other test score.

The first twenty candidate descriptions were learning trials so that subjects could practice with the rating scale and gain experience with the levels of the two attributes. The correlation between the test scores was manipulated on these trials. The test scores in the learning phase was +.80 for half the subjects, and -.80 for half the subjects. The experimental trials followed immediately after the learning trials without a break. Subjects were unaware of the distinction between learning and experimental trials. For the latter set of trials, the design was fully orthogonal (no correlation). Scores for each test were "Unable", 30, 50, 70, or 90, out of 100 in the positive frame condition, or their complements (number incorrect) in the negative frame condition. All subjects rated all candidates in a within-subject design.

Three between-subjects factors were investigated. First, subjects were told that test scores tend to be positively (negatively) correlated, and an explanation for this was given. This story was reinforced by the correlation of test scores in the learning trials. Second, the frame of the scores was manipulated by describing them as the number correct (incorrect) out of 100. Finally, the weight was manipulated by telling subjects that top management was expected to issue a directive that product knowledge (selling technique) should be emphasized in the hiring decision, and a brief explanation of management's reasoning was given. Twenty subjects were randomly assigned to each of the eight cells generated by this 2 (Correlation) X 2 (Weight) X 2 (Frame) design.

RESULTS

As a first step, multiple regression was employed to estimate the effects of sales technique test scores and product knowledge test scores on ratings. The regression coefficients are plotted in Figures 1 and 2, respectively, for ease of comparison and to reduce visual clutter; however, all statistical tests reported below employed subjects' original ratings as the dependent variable. As a test of the weight manipulation, it may be observed in Figure 1 that technique had a greater effect on ratings when subjects were told that management wanted to emphasize technique ova product knowledge. The reverse may be seen in Figure 2, where product knowledge coefficients are higher when knowledge was emphasized.

According to predictions la and lb, the correlation between the two sets of test scores affects the inferences. This effect may be observed in Figures 1 and 2 by comparing the dotted lines, representing incomplete information trials, to the solid lines, representing complete information trials. The crossover interaction indicates that, relative to the complete information trials, the effect of the presented information on the ratings in the incomplete information trials was enhanced in the positive correlation condition (prediction la) and reduced in the negative correlation condition (prediction lb).

Analysis of Variance confirms the significance of this Correlation X Amount of Information interaction, F(1,152) = 114.87, p < 0.001. Follow up tests of the results for each correlation condition confirm prediction 1a for the positive correlation, F(1,76) = 103.56, and prediction 1b for the negative correlation, F(1,76) = 21.36; p < 0.001 for both tests.

According to prediction 2, the inference effect should be greater when the missing information is more important. This result may be observed by comparing the upper pair of lines to the lower pair in Figures 1 and 2. The Correlation X Amount of Information interaction is more pronounced in the lower pair of lines, representing trials in which the missing information is more important, relative to the upper pair, in which the missing information is less important. This Weight X Correlation X Amount of Information interaction is significant at p<0.001, F(1,152) = 23.28.

The mean ratings for all completely described (solid lines) and all incompletely described (dotted lines) alternatives are plotted in Figure 3. The figure allows for a visual comparison of the mean level of the ratings as a function of attribute information, weight and frame.

The third prediction, that trials with missing information (dotted lines) would lead to lower evaluations than trials with complete information (solid lines), was supported; the main effect for Amount of Information was significant at p<0.001, F(1,152) = 142.32. Furthermore, consistent with prediction 4, this bias against missing information was greater when the missing attribute was more important, F(1,152) = 65.686, p<0.001.

Prediction 5 states that framing effects would be observable for the complete information trials, but not for the incomplete information trials. Figure 3 confirms that the solid lines are higher in the positive frame condition than in the negative frame condition, and that the dotted lines are flat across conditions. This Frame X Amount of Information interaction is significant; F(1,152) = 29.09, p<0.001.

Prediction 6 invokes the null hypothesis. A significant Weight X Frame X Amount of Information interaction would have indicated that weak framing effects do operate on the inference, but that they are unobservable unless the missing information is important. The failure of the weight manipulation to produce framing effects in the inference is indirect support for the interpretation that inferences are made after the subjective valuation of presented information.

The one prediction that was not supported dealt with the identification of the locus of the framing effect (prediction 7). Because the frame affects the perception of the presented information, which in turn forms the basis of the inference, it was argued that the framing effect on incomplete trials should be greater when the inter-attribute correlation is positive and smaller when the correlation is negative. The test of this prediction was the Correlation X Frames interaction; F(1,152) = 2.609, p < 0.11. It was expected that this effect would be small, and masked by the effects of the weights of the attributes. Because the sample size of each cell was only 20 and the trend was in the predicted direction, this prediction warrants further study in future investigations.

FIGURE 1

EFFECT OF SALES TECHNIQUE TEST SCORES

FIGURE 2

EFFECT OF PRODUCT KNOWLEDGE TEST SCORES

FIGURE 3

DISCOUNTING AND FRAMING EFFECTS

DISCUSSION

This is the first time that 1) both positive and negative correlations have been tested in the same controlled inference experiment, and 2) the weight of the attributes has been demonstrated to interact in the predicted manner with the correlation and the bias against missing information. Previous investigations have relied on inter-attribute correlations which are presumed to occur in natural settings, but which may not be appropriate for all respondents. In addition, earlier investigations depended on subjects' verbal reports and on indirect inferences about the relative weights of the attributes in order to test the interaction between weight and inferences. This study manipulated correlations and weight in a controlled setting and demonstrated that subjects are capable of the complex information processing implied by the inference model, and that inference effects follow the patterns predicted by the model. The results provide strong support for a simple model which has been shown to accurately predict judgments and decisions concerning ground beef, televisions, clinical psychologists, student performance, and gambles. Similar models have been applied to hiring decisions and ratings of beer.

The inference model has important implications for the conduct of laboratory and market research Investigation of the effects of inferences has demonstrated the need to carefully select attributes to be included in the survey design, because subjects do not use only the information that is given them. Furthermore, if it is impossible to include all the relevant information, the model allows us to predict the effects of the missing information. Finally, the results tentatively suggest that marketing managers may wish to omit attribute information from their advertising in order to exploit inferences that may be more favorable than the factual attribute level.

REFERENCES

Anderson, Norman H. (1981) Foundations of Information Integration Theory. New York: Academic Press.

Anderson, Norman H. and Lola Lopes (1974), Some psycholinguistic aspects of person perception. Memory and Cognition, 2, 67-74.

Birnbaum, Michael H., Rebecca Wong, and Leighton Wong (1976), Combining information from sources that vary in credibility. Memory and Cognition, 4, 330-336.

Huber, Joel and John McCann (1982), The impact of inferential beliefs on product evaluations. Journal of Marketing Research, 19, 324-333.

Johnson, Richard D. and Irwin P. Levin (1985), More than meets the eye: The effect of missing information on purchase evaluations. Journal of Consumer Research, 12, 169-177.

Johnson, Richard D., Irwin P. Levin, and Luanne J. Cressey (1986), Relying on your own best judgment: Imputing values to missing information in decision making. Paper presented at the meeting of the Midwestern Psychological Association, Chicago.

Johnson, Richard D. and Chwee Teck Tan (1987), Getting what you pay for: The effect of missing attributes on product evaluations. Unpublished working paper, University of Alberta, 15 pages.

Levin, Irwin P., Daniel P. Chapman, and Richard D. Johnson (1988), Confidence in judgments based on incomplete information: An investigation using both hypothetical and real gambles. Journal of Behavioral Decision Making, 1, 29-42.

Levin, Irwin P., Richard D. Johnson, Craig P. Russo, and Patricia J. Deldin ( 985), Framing effects in judgment tasks with varying amounts of information. Organizational Behavior and Human Decision Processes, 36, 362-377.

Lim, Jeen-Su, Richard W. Olshavsky, and John Kim (1988), The impact of inferences on product evaluations: Replication and extension. Journal of Marketing Research, 25, 308-316.

Meyer, Robert J. (1981), A model of multiattribute judgments under uncertainty and informational constraints. Journal of Marketing Research, 18, 428-441.

Meyer, Robert J. (1982), A descriptive model of consumer information search behavior. Marketing Science, 1, 93-121.

Yamagishi, Toshio and Charles T. Hill (1981), Adding versus averaging models revisited: A test of the path-analytic integration model. Journal of Personality and Social Psychology, 41, 13-25.

Zwick, Rami (1988), Effects of information presentation format on consumers' inferential beliefs formation. Paper presented at the ORSA/TIMS Marketing Science conference, Seattle.

----------------------------------------