Context Effects in Proxy Judgments

ABSTRACT - In many surveys, respondents are asked to answer questions about both themselves (self reports) and others (proxy reports). In this paper, we report the results of an experiment that examines the conditions under which respondents use their self report in making a proxy judgment. The findings indicate that the distance between the self and proxy report and the type of comparison strategy evoked by an earlier question affect both the weight given to the self report and the convergence between self and proxy reports. These results are consistent with a theoretical framework proposed by Schwarz and Bless (in press). Implications for reducing measurement errors in surveys via questionnaire design are discussed.

Citation:

Barbara Bickart, Geeta Menon, Seymour Sudman, and Johnny Blair (1992) ,"Context Effects in Proxy Judgments", in NA - Advances in Consumer Research Volume 19, eds. John F. Sherry, Jr. and Brian Sternthal, Provo, UT : Association for Consumer Research, Pages: 64-71.

Advances in Consumer Research Volume 19, 1992      Pages 64-71

CONTEXT EFFECTS IN PROXY JUDGMENTS

Barbara Bickart, University of Florida

Geeta Menon, New York University

Seymour Sudman, University of Illinois at Urbana-Champaign

Johnny Blair, University of Maryland

[This project was funded by the National Science Foundation, Grant No. SES-8821362. We thank Lauranne Buchanan, John Lynch, and Stan Presser for helpful comments on an earlier draft of this paper.]

ABSTRACT -

In many surveys, respondents are asked to answer questions about both themselves (self reports) and others (proxy reports). In this paper, we report the results of an experiment that examines the conditions under which respondents use their self report in making a proxy judgment. The findings indicate that the distance between the self and proxy report and the type of comparison strategy evoked by an earlier question affect both the weight given to the self report and the convergence between self and proxy reports. These results are consistent with a theoretical framework proposed by Schwarz and Bless (in press). Implications for reducing measurement errors in surveys via questionnaire design are discussed.

INTRODUCTION

In consumer research, we often ask people to report about both their own behavior and attitudes and those of other people. Reports about others are referred to as proxy reports. For example, one respondent may be asked to answer questions about the purchase and consumption behavior of all household members. Because household members are likely to be similar on these dimensions, a respondent's self report can be a diagnostic, or useful, input to such judgments.

The purpose of this research is to investigate how contextual factors, such as questionnaire format, affect the use of respondents' self reports in making judgments about their partner. In addition, we examine how contextual factors might affect the adequacy of proxy reporting. Specifically, we examine differences in convergence, which is the correlation between the proxy report and the target person's self report.

After a brief review of the relevant literature, we present the results of an experiment that investigates the effects of two contextual factors on (1) the use of one's self report in making a proxy report and (2) the convergence between reports. These factors include the distance between the self and proxy report and the inclusion of questions intended to evoke a comparison basis for making the proxy report. The theoretical and practical implications of these results are then discussed.

LITERATURE REVIEW

In many cases, summary judgments about respondents' own attitudes and behaviors may not be accessible (Lastovicka and Bonfield 1982). For example, while a respondent may go to a restaurant frequently, s/he may not be able to retrieve the number of times s/he visited a restaurant in the last month simply because the information is not accessible in memory. Therefore, it is unlikely that respondents have other people's global judgments regarding various issues and past behaviors stored in memory. Hence, judgments about others are likely to be constructed on the spot, using inputs retrieved from memory.

Hoch (1987) suggests that when making judgments about others, people can use three kinds of information: (1) their own judgment regarding the topic, (2) the perceived level of similarity between themselves and others, and (3) other relevant information, such as prior conversations or observed behavior. For example, Davis, Hoch and Ragsdale (1986) found that people relied heavily on their own preferences in predicting their spouse's preferences for a variety of consumer products. In addition, respondents appeared to use knowledge about their spouse's perceived level of influence in the purchase decision for the product in answering the question. Finally, Davis et al. (1986) found that couples would often be more accurate if they relied completely on their self report and did not adjust it at all based on other information (see also Hoch 1987). This suggests that when people are similar, a self report is a diagnostic input to a report about their partner. Thus, we might want to investigate strategies to encourage the increased use (or weight) of one's self report when making a proxy report.

Questions placed earlier in an interview have been shown to increase the accessibility of information used to answer a later question (see Feldman and Lynch 1988 and Tourangeau and Rasinski 1988 for reviews). Thus, an earlier self report should render self information accessible and increase the likelihood that the self report is used in answering a later question about one's partner. However, Davis, et. al. (1986) varied the order in which self and proxy reports were elicited, and found no effect of question order on the weight given to one's own preference in predicting their partners'.

Earlier questions could also invoke a strategy that respondents may use to answer later items. For example, Menon et al. (1990) had respondents give both self and proxy reports of the favorability of political groups. In some conditions, the proxy report immediately followed the self report, and in other conditions, the items were separated by a series of unrelated questions. In addition, in some conditions a question querying the extent of agreement between the couple on political issues preceded the proxy judgments. They found that respondents were more likely to report using their self report in answering the proxy question when it was preceded by the agreement question. This only occurred, however, when the self and proxy reports were queried contiguously. Menon et al. (1990) suggest that the agreement question invoked a comparison-based strategy that resulted in the use of the self report when it was relatively accessible (e.g., it had been recently queried in an earlier question).

Our research extends this previous work in several ways. First, previous research has dealt only with judgments about purchase intentions and attitudes. We look at judgments about both attitudes and behavioral frequencies. The latter may be especially important in marketing research, since many market share predictions and forecasts are based on consumer reports of how often they purchase and/or use different products. Second, we consider the effects of invoking other types of strategies on the use of one's self report, such as comparisons to a norm. Finally, we explore the relationship between contextual factors and the convergence between the proxy report and the target person's self report.

HYPOTHESES

In this experiment, we attempted to manipulate two factors: (1) the accessibility of the self report, and (2) the comparison basis used in making the proxy judgment. The key dependent measure was the weight given to the self report in making the proxy report. We expected an interaction between the two manipulated variables on this measure. Specifically, invoking a self-based comparison strategy should only affect the weight given to the self report when it is relatively accessible. When the self report is not accessible, invoking a self-based comparison strategy should not affect the weight given to the self report in making the proxy judgment, because the self report is not accessible to the respondent to make this comparison. Assuming that couples are relatively similar in the queried domains, we expect convergence between reports to follow a similar pattern. Thus, convergence should be higher when the self report is accessible and a self-comparison strategy is invoked. Finally, invoking a norm-based comparison strategy should not affect the weight given to the self report under any condition. This is because the self report is not a diagnostic input to a norm-based comparison.

METHOD

This experiment was embedded in a larger study dealing with the cognitive aspects of proxy reporting (see Bickart et al., 1990, Blair, Menon, and Bickart, in press; and Sudman et al., in press). The survey included questions about a variety of topics, including media usage, consumption of alcoholic beverages, health status, political attitudes, and demographics. Most of these questions were taken from other surveys conducted by major survey research institutions. In this paper, we focus on the four questions shown in Figure 1: (1) TV viewing (TV), (2) beer consumption (Beer), (3) an overall health rating (Health), and (4) attitudes toward the federal government's drug policy (Drugs).

Telephone interviews were conducted with 201 couples. A standard directory-based random digit dial sample design was used to select households (Sudman 1973). Couples were eligible to participate if they were either married or living together as married. Agreement to participate was secured from both partners before either was interviewed. Respondents were requested to ask their partner to leave the room during the interview, and not to discuss the survey until both interviews were complete. The interview averaged 20 minutes per person.

Two variables were manipulated in the questionnaire. First, the accessibility of the self report was manipulated by the distance between self and proxy questions. In half of the questionnaires, self reports on all topics were obtained before the proxy reports, creating a buffer between items (the buffer condition). In the remaining questionnaires, the self and proxy questions were obtained sequentially for each topic (no buffer condition).

Second, the comparison basis used in making the proxy report was manipulated by the addition of a question immediately preceding each of the items in Figure 1. We refer to this as a "priming" question, as the intent was to heighten the accessibility of certain comparison standards at the time the proxy report was made. As shown in Figure 2, there were three types of priming questions. First, in some conditions respondents were asked to assess the similarity between their own behavior or attitude and that of their partner. In a second condition, respondents assessed the dissimilarity between themselves and their partner on the topic. In both cases, the intent of the questions was to evoke the use of a strategy suggesting a comparison to the self report. Because there were no differences on the dependent measures when similarities versus dissimilarities were queried, these conditions were combined (the Prime-Similar condition). Third, some respondents were asked to compare their partner's behavior or attitude to "most other people". The intent of this question was to increase the accessibility of a "norm-based" comparison process (the Prime-Norm condition). Finally, no priming question was included in the control condition. Crossing these variables resulted in a 3 x 2 between-subjects design, with two levels of the distance between self and proxy report (buffer/no buffer) and three priming conditions (control, prime-similar, prime-norm). Treatment conditions were randomly assigned to couples. Because the similar and dissimilar conditions were combined, cell sizes varied between 45 and 105 respondents, although the number of observations included in the analysis of individual items may differ due to nonresponse.

RESULTS

The key dependent variable in this analysis was the weight given to the self report in making the proxy report. It was predicted in the no buffer condition, this weight should be greater in the prime-similar condition, compared to the control condition. In the buffer condition, the weight given to the self report should not vary by priming condition. Finally, when norms were primed, the use of the self report should not vary significantly from the control condition.

FIGURE 1

WORDING OF PROXY QUESTIONS

FIGURE 2

WORDING OF PRIMING QUESTIONS

TABLE 1

REGRESSION COEFFICIENTS INDICATING THE EFFECTS OF BUFFER, PRIME, AND SELF REPORT ON PROXY REPORT

Regression analysis was used to test these hypotheses. Separate regression models were estimated using OLS for each of the four key questions. In these models, the proxy judgment was regressed on the self report, one dummy variable representing question order (Buffer = 1, No Buffer = -1), two dummy variables representing the priming conditions, all interactions, and gender as a covariate (Males = 1, Females = -1). The dummy variables for the priming condition were contrast coded to compare each priming condition with the control condition (X1: Control = -1, Prime-Similar = 1, Prime-Norm = 0; X2: Control = -1, Prime-Similar = 0, Prime-Norm = 1) (Cohen and Cohen 1983). Both the proxy judgment and the self report were centered on the mean for this analysis.

TABLE 2

UNSTANDARDIZED REGRESSION COEFFICIENTS REFLECTING THE WEIGHT GIVEN TO SELF REPORT IN MAKING PROXY REPORT ESTIMATED WITHIN CONDITION

The results of the regression analyses are shown in Table 1. For each model, the independent variables accounted for a significant proportion of the variance in the proxy report, ranging from 10% for reports about health status to 61% for reports about drug control policy. Not surprisingly, there was a significant main effect of gender in three of the four models (TV, Health, and Beer). This suggests that the mean level of reported proxy behavior or attitudes varied by gender. For example, women reported higher levels of beer consumption for their husbands. These differences are reflective of differences in self reports (e.g., men reported consuming beer more often than did women in their self reports). (See Bickart et al. 1991 for a further discussion of gender differences in proxy reporting.) In addition, the self report had a large and significant effect on the proxy report in each of the models.

We were most interested in whether the use of the self report varied as a function of the buffer and prime manipulations. This hypothesis is tested by the significance of the beta coefficient for the Self x Buffer x Prime interactions. A significant interaction suggests that the weight given to the self report varies as a function of the buffer and prime condition, relative to the control condition. The results of the regression analyses suggest that priming similarities, but not norms, affects the use of one's self report differently depending on the accessibility of the self report, compared to the control condition. For three of the four items (TV, Beer, and Health) the Self x Buffer x Prime-Similar interaction is significant, while for the fourth item (Drugs) only the Self x Buffer interaction is significant. Priming norms, on the other hand, did not affect the use of the self report for any of the items (p's > .10).

We then examined the nature of the Self x Buffer x Prime-Similar interaction. Our hypothesis suggests that the Self x Prime-Similar interaction should be significant in the No Buffer condition, but not in the Buffer condition. To test this, separate regression models were estimated within each Buffer condition, and the significance of the beta coefficient for the Self x Prime-Similar interaction was examined. Contrary to expectations, the weight given to the self report was more likely to vary as a function of the priming manipulation in the buffer condition (versus the no buffer condition).

These results are summarized in Table 2, which shows the unstandardized regression coefficients estimated within buffer and priming conditions. These coefficients represent the weight given to the self report within these conditions. In the buffer condition, greater weight was given to the self report when similarities were primed for both the TV (E(1,186)=6.18, p<.05) and the Health (E(1,194)=4.17, p<.05) items. For the other items (Beer and Drugs), the trends were in a similar direction, but were not significantly different. It appears that querying similarities increased the accessibility of self information, which had become less accessible due to the buffer of questions.

In the no buffer condition, the impact of the self report on the proxy report varied significantly as a function of the priming condition only for the TV item. In this case, the self report received less weight when similarities were primed (E(1,184)=4.72, p<.05). A consistent pattern occurred for the other items, although the differences were not significant. Thus, contrary to expectations, priming a self-based comparison strategy seemed to result in a reduced use of one's self report in making a proxy report when the self report was accessible.

TABLE 3

CONVERGENCE (CORRELATION BETWEEN PROXY REPORT AND OTHER'S SELF REPORT) BY CONDITION

For the Drug item, the use of the self report varied only as a function of the buffer condition, as indicated by a significant Self x Buffer interaction. In this case, the self report influenced the proxy report more in the no buffer condition, that is when the self information was more accessible. The priming condition had no effect on responses to the drug item, possibly because respondents had not discussed the drug issue with their partner, and thus had little information on which to base their response to the priming question.

Finally, we were interested in seeing if the reduced anchor was associated with lower convergence between reports. Convergence is probably not a good indicant of accuracy in this case, given the social desirability biases that are likely for at least three of the items (TV, Beer, and Health). The convergence correlations, computed within the experimental cells, are shown in Table 3. The difference between correlations in the control and priming conditions within buffer conditions is of key importance. Correlations were transformed using Fisher's r to z transformation prior to significance testing. In the no buffer condition, correlations were significantly lower when similarities were primed compared to the control condition for each of the items. Thus, priming a comparison-based strategy when the self report is accessible appears to be related to reduced convergence. In the buffer condition, priming a self-based comparison does not appear to affect convergence.

There are two important limitations to the convergence measure used here. First, convergence is likely to represent shared error between the self and proxy report (Bickart et al., 1991). Thus, both reports may be biased in the same direction, especially given the social desirability of the items. Second, convergence may also represent the actual similarity between the couple. Thus, differences in convergence can not be directly related to differences in the use of one's self report until actual similarity is considered.

DISCUSSION

There were several key findings in this experiment. First, when the self and proxy report were separated by a series of questions, people gave more weight to their self report when similarities were primed in a question preceding the proxy report. Priming similarities appeared to increase the accessibility of the self report, which may have been reduced due to the buffer.

When the self and proxy reports were elicited sequentially, the weight given to the self report was consistently lower when similarities were primed, compared to the no prime condition. Because the self and proxy reports were elicited sequentially, the self report should be relatively accessible. Thus, in this condition respondents appear to reduce the weight given to the self report. This runs directly counter to the results reported by Menon et al. (1990), who used concurrent protocols as a dependent measure. It is possible that respondents' verbal reports of increased use of self information are a better indication of the accessibility of inputs, versus the actual use of inputs.

Earlier questions may have affected both the accessibility of inputs used to construct a judgment, and the strategy used to form the judgment. If a comparison strategy is activated, the increased use of the self report should be contingent on the similarity of the couple in the given domain. Modal responses on the similarity questions suggest that most respondents felt they were at least somewhat like (not very different from) their partner in the queried domains, with the exception of beer consumption, where people tended to rate themselves as being either very alike or very different. Because the self report was accessible (compared to the buffer condition), the priming question may have increased the salience of any differences between the couple. Thus, the self report may have been given less weight to account for such differences. Future analyses will take this contingency into consideration.

It is also possible that when a comparison strategy was activated and the self report was accessible (e.g., had recently been queried), respondents may have used the self report as a standard of comparison for making the proxy report, regardless of the content of the self report. A framework proposed by Schwarz and Bless (in press) suggests that this is likely to occur when: (a) respondents are aware that the information has come to mind due to a preceding task, and (b) respondents feel that their response will be redundant with earlier answers. The manipulations in this experiment may have produced such effects. Further, the Schwarz and Bless model suggests that if (a) and (b) do not hold, assimilation is likely to occur. Again, this is consistent with the results in the buffer condition, where respondents are less likely to realize that their self report had been rendered accessible due to the context.

Finally, preliminary results from this study suggest that convergence between the proxy report and the other person's self report was lower when a self-based comparison strategy was primed and the self report was accessible. Under these conditions, it appears that respondents may have discounted their self report in making the proxy report. Assuming that the self report is a diagnostic input for the proxy judgment, discounting the self report would result in a less accurate judgment. Because respondents are more likely to use their self report when it is accessible, eliciting self and proxy judgment sequentially may be the optimal strategy. It is important to note that convergence may not be a good indicant of accuracy in this case. High convergence may simply reflect shared error between the self and proxy report, particularly given the sensitive topics queried in this study (Bickart et al. 1991). Further research is needed in this area.

REFERENCES

Bickart, Barbara, Johnny Blair, Seymour Sudman and Geeta Menon (1991), "An Experimental Study of the Effects of Level of Participation on Proxy Reports of Vacation Planning," in Proceedings of the American Statistical Association: Section on Survey Research Methods, Atlanta, GA.

Bickart, Barbara, Johnny Blair, Geeta Menon, and Seymour Sudman (1990), "Cognitive Aspects of Proxy Reporting of Behavior," in Advances in Consumer Research, Vol. 17, eds., M. Goldberg, G. Gorn, and R. W. Pollay, Provo, UT: Association for Consumer Research, 198-206.

Bickart, Barbara, Geeta Menon, Norbert Schwarz, Seymour Sudman and Johnny Blair (1991), "Gender Differences in Proxy Reporting," paper presented at the American Association for Public Opinion Research Meetings, Phoenix, AZ (May).

Blair, Johnny, Geeta Menon, and Barbara Bickart (in press), "Measurement Effects in Self vs. Proxy Responses to Survey Questions: An Information Processing Perspective," in Nancy C. Mathiowetz (ed.) Measurement Errors in Surveys. John Wiley Press.

Cohen, Jacob and Patricia Cohen (1983), Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Davis, Harry L., Stephen J. Hoch, and E.K. Easton Ragsdale (1986), "An Anchoring and Adjustment Model of Spousal Predictions," Journal of Consumer Research, Vol. 13(June) 25-37.

Feldman, Jack M. and John G. Lynch, Jr. (1988), "Self-Generated Validity: Effects of Measurement on Belief, Attitude, Intention and Behavior," Journal of Applied Psychology. 73 (August), 421-435.

Hoch, Steven J. (1987), "Perceived Consensus and Predictive Accuracy: The Pros and Cons of Projection, "Journal of Personality and Social Psychology, 53 (2), 221-234.

Lastovicka, John L. and E.H. Bonfield (1982), "Do Consumers Have Brand Attitudes?", Journal of Economic Psychology, 2, 57-75.

Menon, Geeta, Seymour Sudman, Barbara Bickart, Johnny Blair, and Norbert Schwarz (1990), "The Use of Anchoring Strategies by Proxy Respondents in Answering Attitude Questions," paper presented at the American Association for Public Opinion Research Meetings, Lancaster, PA (May).

Schwarz, Norbert and Herbert Bless (in press), "Constructing Reality and Its Alternatives: An Inclusion/Exclusion Model of Assimilation and Contrast Effects in Social Judgment," in L.L. Martin and A. Tesser (Eds.), The Construction of Social Judgment. Hillsdale, NJ: Erlbaum.

Sudman, Seymour (1973), "The Uses of Telephone Directories for Survey Sampling," Journal of Marketing Research, 10, 204-207.

Sudman, Seymour, Barbara Bickart, Johnny Blair, and Geeta Menon (in press), "A Comparison of Self and Proxy Reporting," in Norbert Schwarz and Seymour Sudman (eds.), Autobiographical Memory and the Validity of Retrospective Reports, New York: Springer-Verlag.

Tourangeau, Roger and Kenneth A. Rasinski (1988), "Cognitive Processes Underlying Context Effects in Attitude Measurement," Psychological Bulletin, Vol. 103, No. 3, 299-314.

----------------------------------------