Cognitive Aspects of Proxy Reporting of Behvior

Barbara A. Bickart, University of Florida
Johnny Blair, University of Illinios
Geeta Menon, University of Illinois
Seymour Sudman, University of Illinois
[ to cite ]:
Barbara A. Bickart, Johnny Blair, Geeta Menon, and Seymour Sudman (1990) ,"Cognitive Aspects of Proxy Reporting of Behvior", in NA - Advances in Consumer Research Volume 17, eds. Marvin E. Goldberg, Gerald Gorn, and Richard W. Pollay, Provo, UT : Association for Consumer Research, Pages: 198-206.

Advances in Consumer Research Volume 17, 1990      Pages 198-206

COGNITIVE ASPECTS OF PROXY REPORTING OF BEHVIOR

Barbara A. Bickart, University of Florida

Johnny Blair, University of Illinios

Geeta Menon, University of Illinois

Seymour Sudman, University of Illinois

[This project was funded by the National Science Foundation, Grant No. SES-8821362. We would like to thank Norbert Schwarz for his useful comments on an earlier version of this paper.]

INTRODUCTION

This paper presents some preliminary results from a study being conducted by us at the Survey Research Laboratory, University of Illinois and funded by the National Science Foundation. We are investigating the cognitive processes that underlie self and proxy reporting in surveys and their impact on data quality. We are particularly interested in the quality of data provided by proxies and explore possible ways to improve the quality of proxy reports.

In many surveys conducted by government agencies, universities, and the private sector, respondents must provide data about themselves, other household members and sometimes about the household and the community in which they live. For example, the Current Population Survey requires that one household member report about the labor forces participation of all household members; in the Health Interview Survey and the National Crime Survey proxy respondents are permitted to report about other household members if they are not available. In consumer research, information about multiple family members' brand purchases and usage is often obtained using proxies because of research budget constraints.

Prior work examining the adequacy of proxy reports has usually found that self reports are more complete. However in some situations small differences are found and, in a few cases, proxy reporting is actually better. (For a review see Moore, 1988.) Many of the comparisons reported in the literature have been flawed because of possible selection biases. Typically, proxies are used only when the respondent is unavailable, thus undermining systematic comparison. In addition, earlier studies have not provided any understanding of how the cognitive processes used for proxy and self-reporting might differ. Therefore, these studies have been unable to use an understanding of processing differences to suggest methods to improve proxy reporting.

Our study addresses these limitations by studying pairs of respondents in a household and asking them to report both about themselves and another household member. We use divergent methodologies, including think-aloud interviews in a cognitive laboratory setting, telephone interviews and laboratory experiments, and our studies include survey questions on a wide range of topics, as shown in Table 1. Our research program focuses on the following major issues:

1. The convergence between self and proxy reports of behavioral frequencies and the factors that influence convergence, particularly the joint participation of the pair in the activity and the importance of the behavior to the respondent;

2. The use of anchoring and adjustment strategies by proxies to answer both behavioral frequency and attitude questions;

3. The different processes used by self and proxy respondents to answer behavioral frequency questions.

Our current paper presents some preliminary results related to this last issue. In what follows, we give a theoretical overview of current thinking on how people respond to behavioral questions and why proxy reporting processes might be expected to differ. This is followed by a brief methodological discussion, the presentation of our results and a concluding section on the meaning of the results and plans for future research.

THEORETICAL BACKGROUND

A behavioral frequency question requires the respondent to identify the target behavior, to determine if it occurred at all during a specified time period, and if so, how often (see Schwarz, in press, for a detailed discussion of these tasks). Traditionally, it has been assumed that respondents determine the frequency of a behavior using a "recall and count" procedure. Presumably, they search memory for relevant behavioral episodes, determine the date of these episodes, and count the number of occurrences within the reference period. Unfortunately, recent research suggests that this apparently straightforward procedure is rarely used. Specifically, it is most likely to be used for rare and outstanding events, but unlikely to be used for frequent and mundane behaviors as they are usually assessed in surveys. Most importantly, frequent and mundane behaviors--such as reading the newspaper or eating out--are not represented as separate episodes in memory. Rather, these highly similar episodes tend to blend into one generic representation of the behavior, making the use of a recall and count strategy infeasible. Accordingly, frequency reports of mundane behaviors are usually based on estimation procedures in which respondents use some fragmented recall of behavioral episodes along with various inferential rules to derive an appropriate estimate. Which strategy they use is likely to depend on how the information is organized in memory.

TABLE 1

TOPICS COVERED IN STUDY OF PROXY REPORTING

Table 2 present a heuristic framework, suggested by Geeta Menon (1989) as part of her dissertation research, that conceptualizes the impact of different memory representations on the likely use of different recall and estimation strategies. This model posits that there are two key factors that influence how information about an autobiographical event is stored in memory. The model further suggests that the specific form of the memory representation influences the strategy that respondents use to retrieve information to form a behavioral frequency judgement. The first factor pertains to whether the event occurs at regular or irregular intervals. We refer to this as the regularity or periodicity of the behavior. The second factor pertains to whether separate occurrences of the behavior are similar or dissimilar to one another. We refer to this factor as the variance of the behavioral episodes.

The model assumes that behaviors with a highly regular pattern of occurrences may be represented in memory in a summary form that reflects the respondent's semantic knowledge about his or her usual behavior. This representation may often have the form of a rate-of-occurrence (e.g., "I read the newspaper daily"). Accordingly, respondents may retrieve the rate-of-occurrence for the target behavior and may base their estimate on the rate. For example, a respondent who knows that she reads the newspaper daily would easily come up with an estimate of "seven" for the referenced period of one week, without recalling a single episodic representation of "reading the newspaper". Moreover, this initial estimate may be easily adjusted if the respondent does not read a Sunday paper, and so on.

In contrast to regularly occurring behaviors, however, behaviors with an irregular pattern of occurrence are unlikely to be represented in a summary form, such as a rate-of-occurrence. Accordingly, respondents will have to use some kind of episodic recall to compute an estimate under irregular occurrence conditions. The accessibility of individual episodes in memory, however, is likely to depend on the variance of the behavior. As several studies have shown, similar occurrences tend to blend into--one generic representation, rendering the individual instances difficult to retrieve. Distinct occurrences, on the other hand, are represented as separate episodes, thus increasing the likelihood that they can be recalled (Linton, 1982; White, 1982; Wagenaar, 1986). Thus, the more variant the behavior (e.g., purchases of major household or personal items) the more likely that respondents will be able to access independent episodes. In that case, they may either count these episodes and report the frequency, or use a subset of them to form a rate on the basis of which to make a frequency judgement.

On the other hand, the more invariant the behavior (e.g., reading the same newspaper in the same setting, but on irregular days), the more likely that respondents will not be able to access individual episodes. Accordingly, they will have to estimate a behavioral frequency on the basis of general knowledge about their behavior. This, however, is likely to be particularly difficult because the semantic representations, such as "I sometimes read the paper", offer little frequency information over and above eliminating the possibility of a nonoccurrence.

In summary, the regularity and variance of a behavior are expected to interact in determining the strategy that respondents use in providing self reports of behavioral frequencies. In addition to these fairly complex interactions, reporting about the behavior of others raises additional difficulties.

TABLE 2

PROPOSED MODEL FOR TEH STORAGE AND RETRIEVAL OF INFORMATIONABOUT FREQUENT BEHAVIOR

Table 3 summarizes the major differences in the encoding and organization of information about one's own behavior as opposed to information about the behavior of others.

At the encoding level, it is important to note that one's own behaviors provide a rich set of experiences, including information about what we wanted to do, what we actually did, how we felt while doing it, and so on. Thus, the episodic representation is likely to include information relevant to the event, such as the location, and the emotional responses to the event (Tulving, 1972, 1983). In contrast, proxy respondents are typically answering questions about reported events, or events that are learned "second-hand." These events are likely to be represented as episodes which relate to the occasion of receiving or learning about the event. In line with this assumption, Larsen and Plunkett (1987) found that information about reported events was accessed through the memory of the context in which the respondents learned about the event. These considerations have several implications for the strategies used to answer behavioral frequency questions.

First, cues related to the event itself should be more effective in enhancing recall for self reports than proxy reports.

Second, reported events might not be encoded in chronological order. Therefore, proxy respondents should be less likely to use a chronological pattern (i.e., forward or backward search of memory) when searching a reference period.

Finally, this suggests that the similarity between a respondent pair's episodic representation of an event should be related to their common participation in the behavior. Therefore, the similarity of reporting strategies should be directly related to joint participation in the event.

TABLE 3

INFORMATION ABOUT SELF AND OTHER AT STAGES OF

INFORMATION PROCESSING

In addition to these encoding differences, the organization of information stored about oneself versus others might differ for several reasons. First, self-relevant information has been shown to receive increased elaboration at encoding (e.g., Kuiper and Rogers, 1979; Rogers, Kuiper, and Kirker, 1977). This is thought to create a more elaborate memory trace for the event, resulting in enhanced recall. Thus, autobiographical events should be more likely to receive increased elaboration, and episodic representations of one's own behavior should be more accessible. Self-reports should therefore be more likely to be based on the use of a recall and count strategy than proxy reports. Moreover, the attention that is given to the behavior of others may be a function of the other person's importance. As several studies have shown, information that is relevant to important others (e.g., one's spouse) also receives increased elaboration (Bower and Gilligan, 1979; Kuiper and Rogers, 1979). Therefore, it would be expected that the social distance between the self and proxy should be a major factor in determining the similarity in reporting strategies, particularly when the behavioral frequency question involves reporting of relatively variant, and/or infrequent behaviors.

The most pronounced difference in terms of self/proxy reporting may occur when answering a frequency question that requires the retrieval of a rate, as opposed to specific events. Recent research in social cognition has shown that recall of descriptive information, such as a rate, is facilitated when it pertains to the self (Klein and Kihlstrom, 1986; Klein, Loftus and Burton, 1989). This is thought to be attributable to a better organizational structure of this information in memory (Klein and Kihlstrom, 1986). Therefore, rate-based estimates should be more accurate when made about oneself than about others. For proxies, a question which requires the retrieval of a rate may be the most difficult of all to answer and the least accurate.

Finally, the impact of social desirability and self-presentation concerns should be different for self and proxy respondents. Although proxy respondents may, in general, be more willing to report about socially undesirable behaviors of others, they may lack the knowledge to do so. Again, a major factor should be the social distance between the pair. This should affect both the information which a proxy respondent has about the other, and his or her willingness to report it. However, these effects may actually counteract one another, and therefore may be difficult to detect.

These differences are summarized in Table 3. Based on this table, it is suggested that two key variables, the social distance between the pair and the extent to which the pair participated in the behavior together, will mediate the processes used by proxy respondents in answering behavioral frequency questions. Although these variables are not the focus of the analysis which follows, they play a central role in the remainder of this research project.

METHODOLOGY

The results presented today are based on twenty-five pairs of interviews conducted with husbands and wives or couples living together. The interviews were conducted by the senior project staff and covered the topics listed in Table 1. Our discussion is limited to the behavioral items.

We asked respondents the same questions about themselves an the other member of the pair and we asked them to think aloud as they responded. As other users of the think-aloud method have reported (Ericsson and Simon, 1980; Bishop, 1986), respondents vary in their ability to think aloud-some are very verbal and others much less so. If respondents did not think aloud spontaneously, they were asked immediately after answering how they arrived at their answer.

The data were coded by the senior staff using the coding scheme given in Table 4. This coding scheme underwent significant revisions as the staff coded the same interviews and attempted to reconcile differences. The coding frame is still undergoing some revision.

The major variables we examined were:

1. Whether answers were based on counting or estimation methods or a combination:

2. If based on counting, what method was used to search the reference period;

3. If based on estimation, is the estimation rate based, enumeration-based or some combination;

4. What cues were used to retrieve events?

RESULTS

Based on the literature review and the theoretical discussion, we developed four hypotheses:

1. Proxy respondents will be less likely to use chronological sequences than self-reporters;

2. Proxy respondents will be less likely than self reporters to use event cues;

3. Proxy respondents will be more likely than self reporters to use estimation modes for questions of the same level of complexity;

4. Proxy respondents will be less likely to consider the reference period than will self-reporters.

The comparisons between self and proxy responses are presented in Tables 5 and 6. Table 5 gives the average number of responses by method used. This is simply the total responses divided by fifty.

Overall, the obtained findings are nicely consistent with our hypotheses. As may be seen from Table 5, the use of chronological sequences is fairly low for self as well as proxy reports. However, consistent with our Hypotheses One and Two, self reporters are about 60 percent more likely to use chronological sequences and event cues than are proxy respondents.

Hypotheses Three, that proxy respondents are more likely to use estimation methods, is also supported. Table S shows that this is true in total, while Table 6 indicated that the same relation holds for ten of eleven specific topic areas.

Finally, Hypothesis Four, regarding the use of the reference period, also receives some weak supported. Specifically, self reporters do use a specific reference period slightly more than do proxy respondents, but it is clear that this is not a widely used memory strategy by either group. These differences might have been an artifact of lower reporting by proxies and there is some indication of this. Total codes are slightly higher for self (16.22) than proxy (14.88) and if one omits the no-event reported category, there is an average of about one more code per self than proxy.

If proxy respondents use event cues and chronological sequences to a lesser degree than self reporters, one may wonder how they do arrive at an answer? As indicated in Table 6, proxy respondents are significantly more likely to rely on anchor and adjust strategies than self reporters. Specifically, they use their own behavioral frequency as a starting point, and adjust this estimate on the basis of their knowledge about differences between self and other. However, it might be argued that this dominant use of an anchor and adjust strategy reflects the particular question order used in the present study. That is, respondents were always asked first about themselves, before being asked about others. Thus, self related information was rendered easily accessible, making the use of an anchor and adjust strategy particularly easy. This question order, however, reflects the standard procedures used in surveys, thus increasing the relevance of the present findings to current standard survey practice.

TABLE 4

CODING SCHEME

DISCUSSION

The results presented here are the first from a series of investigations that we plan about proxy reporting from a cognitive perspective. It is encouraging that the results, limited as they are, do fit the theoretical framework that we established before data collection.

It is also encouraging that our methods for obtaining think-aloud protocols and coding them seem to be generating useful data. While it can not be ruled out that the think-aloud process itself may affect cognitive processing, we consider the present findings to provide encouraging evidence for the heuristic potential of this methodological approach (Ericsson & Simon, 1980). Given the strength and plausibility of these preliminary findings, we assume that they will hold up-after completion of this study, which will involve a larger sample size and an improved coding scheme. The validity of our conclusions about the strategies of self and proxy reporters will then be the focus of our subsequent work, which will ultimately involve explicit manipulations of these strategies in tightly controlled experiments. We also plan to include the extent of joint participation of a pair in a behavior as a factor to be considered in our analyses. For the time being, we are optimistic that this research will eventually result in improved assessments of self and proxy reports.

TABLE 5

METHODS USED FOR RESPONDING TO BEHAVIORAL QUESTIONS

TABLE 6

COUNTING VS. ESTIMATION BY TOPIC

REFERENCES

Blair, Edward and Scot Burton (1987), "Cognitive Processes Used by Survey Respondents in Answering Behavioral Frequency Questions", Journal of Consumer Research, 14 (September), 280-288.

Bishop, George (1986), 'Think-Aloud Responses to Survey Questions: Some Evidence on Context Effects", paper presented at the NORC Conference on Context Effects in Surveys, Chicago, IL.

Bower, Gordon H. and S. G. Gilligan (1979), "Remembering Information Related to One's Self', Journal of Research in Personality, 13, 420-432.

Ericsson, K. Anders and Herbert A. Simon (1980), "Verbal Reports as Data", Psychological Review, 87 (May 3), 215-251.

Klein, Stanley B. and John F. Kihlstrom (1986), "Elaboration, Organization, and the Self-Reference Effect in Memory", Journal of Experimental Psychology: General, 11,115 (1), 26-38.

Klein, Stanley B., Judith Loftus and Holly A. Burton (1989), "Two Self-Reference Effects: The Importance of Distinguishing Between Self-Descriptiveness Judgments and Autobiographical Retrieval in Self-Reference Encoding", Journal of Personality and Social Psychology, 56, 833-865.

Kuiper, Nicholas A. and Timothy B. Rogers (1979), "Encoding of Personal Information: Self-Other Differences", Journal of Personality and Social Psychology, 37, 499-514.

Larsen, Steen F. and Kim Plunkett (1987), "Remembering Experienced and Reported Events", Applied Cognitive Psychology, 1, 1526.

Linton, Marigold (1982), "Memory for Real-World Events", in Explorations in Cognition, eds. Donald A. Norman and David E. Rumelhart, San Francisco: W. H. Freeman and Company, 366404.

Menon, Geeta (1989), doctoral dissertation proposal, Department of Business Administration, University of Illinois, Champaign, IL 61820.

Moore, Jeffrey C. (1988), "Self/Proxy Response Status and Survey Response Quality", Journal of Official Statistics, 4 (2), 155-172.

Rogers, Timothy B., Nicholas A. Kuiper and W. S. Kirker (1977), "Self-Reference and the Encoding of Personal Information", Journal of Personality and Social Psychology, 35, 677-688.

Schwarz, Norbert (in press), "Assessing frequency reports of mundane behaviors: Contributions of cognitive psychology to questionnaire design." In Research methods in Personality and Social Psychology (Review of Personality and Social Psychology, Vol 11), eds. Clyde Hendrick and Margaret S. Clark, Beverly Hills: Sage.

Tulving, Endel (1972), "Episodic and Semantic Memory", in Organization of Memory, eds., Endel Tulving and Wayne Donaldson, New York: Academic Press, 381-403.

Tulving, Endel (1983), Elements of Episodic Memory, New York: Oxford University Press.

Wagenaar, William A. (1986), "My Memory: A Study of Autobiographical Memory Over Six Years", Cognitive Psychology, 18, 225-252.

White, Richard T. (1982), "Memory for Personal Events", Human Learning, 1, 171-183.

----------------------------------------