Psychometric Characteristics of Behavioral Process Data: Preliminary Findings on Validity and Reliability

Jacob Jacoby, Purdue University
Robert W. Chestnut, Columbia University (student), Purdue University (student), Purdue University (student), Purdue University
Wayne D. Hoyer,
David A. Sheluga, Michael J. Donahue,
ABSTRACT - Along with verbal protocols and eye movement techniques, a family of behavioral process methods have recently been introduced to study pre-decision information acquisition and usage. Various questions may be raised in regard to these methods, including those relating to validity and reliability. Data are presented here regarding the validity and reliability of behavioral process methods. A fuller report will be provided in Jacoby and Chestnut (in preparation).
[ to cite ]:
Jacob Jacoby, Robert W. Chestnut, Wayne D. Hoyer, and David A. Sheluga, Michael J. Donahue (1978) ,"Psychometric Characteristics of Behavioral Process Data: Preliminary Findings on Validity and Reliability", in NA - Advances in Consumer Research Volume 05, eds. Kent Hunt, Ann Abor, MI : Association for Consumer Research, Pages: 546-554.

Advances in Consumer Research Volume 5, 1978      Pages 546-554

PSYCHOMETRIC CHARACTERISTICS OF BEHAVIORAL PROCESS DATA: PRELIMINARY FINDINGS ON VALIDITY AND RELIABILITY

Jacob Jacoby, Purdue University

Robert W. Chestnut, Columbia University

Wayne D. Hoyer (student), Purdue University

David A. Sheluga (student), Purdue University

Michael J. Donahue (student), Purdue University

ABSTRACT -

Along with verbal protocols and eye movement techniques, a family of behavioral process methods have recently been introduced to study pre-decision information acquisition and usage. Various questions may be raised in regard to these methods, including those relating to validity and reliability. Data are presented here regarding the validity and reliability of behavioral process methods. A fuller report will be provided in Jacoby and Chestnut (in preparation).

INTRODUCTION

As noted elsewhere (Jacoby, 1976, p. 5), all that is usually required for a measure or measurement approach to gain acceptance in that consumer research community is for an investigator to describe and use the measure in a single published study. As long as the measure or approach has a certain degree of face validity -- and oftentimes, even if it does not -- it tends to be uncritically accepted.

This uncritical attitude has led to a profusion of operational measures for virtually all of our core constructs. As an example, Jacoby and Chestnut (in press) identify 55 different brand loyalty indices in the published literature and came across several others in the unpublished proprietary literature. The problem, as Kohn and Jacoby (1973) demonstrated in regard to operationally defining innovators, is that different measures of the same construct, when applied to the same subject population, often produce conflicting results. Under these circumstances -- namely, a welter of different definitions which yield contradictory results -- it becomes exceedingly difficult to compare and integrate findings from different investigations. The end result is that consumer research is being strangled by its poor measures and progress in understanding consumer behavior is thwarted. Clearly, we must begin to devote greater empirical attention to describing the psychometric properties of our measures and measurement approaches, particularly the validity and reliability of said measures and approaches. This becomes especially true where we have the possibility of rejecting such measures before they can establish a toehold and proliferate.

Having advocated this in general (Jacoby, 1976; Jacoby and Chestnut, in press), it would be hypocritical if we did not apply this logic in our own research. In particular, since 1972-73 (see Jacoby, 1975a) we have been involved in the development and extension of an empirical approach for the purposes of identifying pre-decision information acquisition. An overview of approximately 20 studies in this series is described in Jacoby (1977) and theoretical underpinnings have been outlined in Chestnut and Jacoby (1977a). More recent developments include: several papers published or in press (Jacoby, Chestnut, and Fisher; Jacoby, Chestnut, and Silberman; Sheluga and Jacoby); completed studies on concept learning (Donahue and Jacoby, in preparation), attribution theory (Major and Chestnut, in preparation), and salesman influence in life insurance purchases (Chestnut, 1977); an illustration of its application to health care services (Chestnut and Jacoby, 1977b); the completion of a monograph on consumer nondurable purchases (Jacoby and Chestnut, 1977); and an on-going project for the Federal Trade Commission (Jacoby and Chestnut, in progress). Much of this work is now being integrated into a single volume (Jacoby and Chestnut, in preparation). [An approach similar to ours, was independently arrived at by Payne (e.g., 1976).]

The approach has also stimulated acceptance and usage by others. Examples from academia include: Bettman (e.g., Bettman and Kakkar, 1977), who was introduced to the approach during a 1975 visit to Purdue and through his participation as a consultant (Bettman and Jacoby, 1976) on the Jacoby and Chestnut (1977; Jacoby, Chestnut, Weigl, and Fisher, 1976) project; Van Raaij (1977a, b), who became acquainted with the approach through a colloquium (Jacoby, 1975b) at the University of Tilburg in May of 1975; and investigators in Mannheim, West Germany where the senior author spent the summers of 1975 and 1976 as a visiting scholar (see Raffee et al., 1976). Further, a variety of commercial applications emanating from this research program have been developed and refined, and are now being offered by the Opinion Research Corporation of Princeton, New Jersey.

Given the burgeoning interest and activity in this approach, it is incumbent upon those who utilize it to begin supplying data regarding its psychometric properties. The present report focuses on validity and reliability, bringing together some unpublished data from several investigations which bear on these issues. [The data on a third psychometric property, sensitivity, will not be detailed in this report. Suffice it to say that BP is believed to provide much greater sensitivity, especially in regard to the sequence of information accessing and usage. Particularly in a crowded product category (where there may be 20 or more different purchase options and 30 or more items of objective information for each), it would seem beyond the capabilities of most people to be able to accurately recall the precise amount, identity, and exact sequence in which information was accessed and used.]

VALIDITY

Addressing validity requires an understanding of the basic kinds of behavioral process (BP) data generated and how these relate to the criteria available. Reduced to its essential core, the BP approach provides data regarding (1) information acquisition behavior -- the information acquired prior to arriving at a decision (which may be further decomposed into the depth, content, and sequence of information acquisition), and (2) choice behavior -- the identity of the option selected when the decision is reached. Validity assessment in these two cases assumes different forms because of the nature of the criteria available.

In the case of information acquisition behavior, there is no external "true value" criterion to serve as a standard and against which we can compare our data. That is, we have no way of crawling into the person's head and directly measuring the information that he acquires in vivo when arriving at a decision in the "real world." Thus, regardless of whether we solicit data using traditional verbal reports (via questionnaire or interview, e.g., "Tell me what information you are acquiring and which information you are ignoring when you make your breakfast cereal purchase decision"), employ verbal protocols (e.g., Bettman, 1970), use a behavioral process approach, or employ an eye tracking procedure (e.g., Russo and Rosen, 1975), we have no ultimate criterion (or "true value") against which to evaluate our data. This is reflected in Figure 1 which depicts verbal reports (VR) and behavioral process (BP) as alternative approaches for assessing actual pre-decision information acquisition. Since we have no way to determine actual information acquisition during real world decision-making, relationships 1 and 2 are not ascertainable. What is measurable is the relationship between the verbal report and BP approach, Relationship 3. This is termed "convergent validity" and represents the necessary first step for demonstrating construct validity (see Campbell and Fiske, 1959; Nunnally, 1967). [The various types of validity and their relationship to each other are discussed in Jacoby and Chestnut (in press, Chapter 4).]

FIGURE 1

RELATIONSHIPS EXISTING BETWEEN VERBAL REPORT AND BEHAVIORAL PROCESS MEASURES OF BEHAVIOR (EITHER INFORMATION ACQUISITION/USAGE OR CHOICE) AND ACTUAL IN VIVO BEHAVIOR

In contrast, in the case of choice behavior, both verbal reports of choice (e.g., "I usually buy Rice Krispies") and the identity of the option selected at the end of a BP simulation are potentially verifiable against an external criterion. That is, if Mrs. Jones says she usually buys Rice Krispies (in response to soliciting a VR), or ends up selecting that brand in a BP simulation, then (at least in theory) it is possible to verify this fact. Hence, all three relationships depicted in Figure 1 are potentially ascertainable for choice behavior.

Information Acquisition Behavior

Convergent validity represents the degree to which different operational measures of the same concept yield comparable results (Relationship 3 in Figure 1), i.e., "a confirmation by independent measurement procedures" (Campbell and Fiske, 1959, p. 81). Data on the convergence between VR and BP measures of information usage are available from several investigations. The earliest example comes from a study conducted during the spring of 1973. In this investigation, Jacoby, Szybillo, and Busato-Schach (1977, p. 213) report a median correlation of .37 (between verbal reports of typical information usage behavior and the order in which these types of information were accessed in a BP simulation) for students reaching a brand choice decision under a Brand Names Present (BNP) condition, and .53 for students arriving at choices under a Brand Names Absent (BNA) condition. [The Brand Names Absent (BNA) manipulation is used to reduce the impact of relevant information in memory on information acquisition behavior. In essence, it makes the BP simulation comparable to a situation in which the subject is confronted with a set of purchase alternatives consisting entirely of "new" brands.] A replication conducted two years later in Germany (Raffle et al., 1976) reveals slightly higher coefficients when VRs of typical usage behavior were correlated with actual usage/non-usage in the BP task: a median of .63 in the BNA condition and .61 in the BNP condition (see Jacoby, Hefner, Sch÷ler, Grabicke, and RaffTe, 1976, p. 9).

Probably the most comprehensive evidence on convergent validity was obtained in an investigation (Jacoby and Chestnut, 1977) [This study, hereafter referred to as the NSF investigation, was supported by a grant from the National Science Foundation (GI-43687).] which employed sociodemographically heterogeneous samples of residents from Tippecanoe County, Indiana, three different test products, and a much more advanced and sensitive BP methodology than was used in the two above-cited investigations. Table 1 presents the data for the information dimensions used by the 456 (out of the 615) subjects in this study who were information seekers. (N.B. The other 149 subjects made their brand choices on the basis of seeing only the brand names of the 16 available brands and did not acquire any additional information prior to arriving at their decision.) As can be seen, the median Spearman coefficients (between self reports of typical information accessing behavior and behavior displayed in the BP decision task) across all information dimensions in the five samples ranged from a low of .22 to a high of .33. Again, values were higher for the BNA than for the BNP condition. Since analysis of the depth of search for these subjects (see Table 8 discussed below) revealed that the mean number of different information dimensions considered prior to making their purchase decisions was 3 to 4 in the BNP condition and approximately 7 in the BNA condition, median Spearman coefficients were identified when only the five most heavily considered dimensions for each of the five samples were considered. These coefficients are: .29 for breakfast cereal; .55 for margarine, BNP; .37 for margarine, BNA; .45 for headache remedies, BNP; and .40 for headache remedies, BNA. Thus, convergent validity seems to be higher (near .40) if attention is limited to frequently used dimensions. Also of interest, for the first time, BNP coefficients are higher than their BNA counterparts.

TABLE 1

RELATIONSHIP OF SELF REPORTS OF TYPICAL ACCESSING BEHAVIOR WITH ACTUAL BEHAVIOR DISPLAYED IN DECISION TASK

In sum, three findings are noteworthy in regard to the convergent validity between VR and BP measures of information acquisition/usage. (1) In general, it would appear that the information people say they typically acquire and the information they actually do acquire in a behavioral process simulation tends to be positively related, but only to a small degree. Coefficients of Determinance (r2) reveals that only between 5% to 40% of the variance is explained. If we omit the two samples using an earlier and less sensitive BP variant of methodology, the remaining Coefficients of Determinance based on full samples and all available information dimensions all fall below 12%. (2) Higher correlation coefficients are obtained when brand names are absent relative to when they are present. This makes considerable sense since, when brand names are present, the individual may be accessing information from memory as well as from the external information environment. Unfortunately, BP procedures can only directly assess information acquired from the external environment. When brand names are absent, all brand-specific information must be acquired from the environment. Thus, without memory to rely on, there should be a greater correspondence between what people say is important to them and what they pay attention to (i.e., access) in a BP simulation. (3) Higher coefficients are obtained for the more heavily accessed information dimensions. Again, this makes considerable sense: people can be expected to less accurately recall accessing infrequently used rather than frequently used information -- especially for low-priced nondurables where much of the information is of a relatively trivial nature.

Several additional comments should be made in regard to the low (r = .2 to .3) correlations obtained between VR and BP indicants when all available information is included and the newer methodological variant is used. First, these coefficients could have been negative, in which case we would have had a major dilemma on our hands (i.e., since there is no available external criterion, which of the two measures do we say provides the "reasonably correct" picture?). On the other hand, these two approaches could have yielded indicants that were in very high agreement (say, r = .8 or .9). In such a case, it could be argued that the BP measures do not provide much unique coverage and (given that they are more difficult, expensive, time-consuming, etc., to collect) should therefore be dispensed with. From this perspective, coefficients in a low to mid positive range (i.e., .3 to .7) would seem to be optimal.

Even granting this argument, the obtained coefficients still show a relatively low degree of relationship and suggest that the measures do contain a fair share of error. However, these low coefficients also seem to be at least partially attributable to factors above and beyond unexplained error. In responding to the VR measure, subjects may have indicated what their behavior may have been like at an earlier point in time but, perhaps (aided by memory), they no longer do. Further, the VR measure asked respondents to indicate what they do "in general," i.e., across purchase instances. In contrast, the BP situation examines only a single instance. As argued elsewhere (Jacoby and Chestnut, in press) in regard to brand loyalty, if we are trying to describe a person's customary behavior, then sampling considerations suggest that we need to consider his behavior across several rather than a single test instance. Stated in a somewhat different form, the two assessments (i.e., VR and BP) were measuring slightly different things: the VR measure asked the individual to indicate what he did in general, whereas the BP measure considered only a single instance. In a similar vein, the BNA condition and the VR data also address somewhat different things. Whereas both the VR measures and BP-BNP condition attempt to reflect and examine the external information environment as it exists in the real world, the BNA condition, through the very absence of familiar brand names, does not. Again, to the extent that different things were being addressed by the two measures (i.e., VR and BP-BNA), we would expect this to be reflected in a lowered correlation. Thus, considering the various actual and potential influences on these coefficients, the values obtained seem quite reasonable and in line with what might have been expected.

Choice Behavior

Both BP and VR methods may be used to collect choice data. Important questions are: (1) just how much agreement exists between these two approaches (Relationship 3 in Figure 1), and (2) how accurate are BP-derived estimates of actual choice behavior (Relationship 2) relative to VR-derived estimates of this same behavior (Relationship 1)? Respectively, these are questions of convergent and criterion-related validity.

Data regarding both types of validity comes from the three BNP samples in the previously noted NSF investigation (Jacoby and Chestnut, 1977). In a fashion similar to numerous other investigations, each subject was asked to recall the names of all brands in the product category he had purchased during the preceding 12 months, and to then estimate the percent of purchases he devoted to each of these brands. The sum of the estimated percent of purchase for each brand was then divided by the total n in each sample to arrive at an aggregated VR estimate of each brand's market share for this sample. Since brands other than those included in our simulation were recalled (e.g., according to Time magazine [1976, p. 61}): "there are no fewer than 156 brands" of breakfast cereal available in American supermarkets and our study considered only 16 of those with the largest market shares), the VR market share estimates cumulated across all brands in our investigation sum to less than 100%. The proportion of time each brand ended up being "purchased" at the end of the BP simulation task represented that brand's BP-derived market share.

Since BP estimates were based upon only those 16 brands provided in each of the decision simulations, these estimates summed to 100% for each product category. In contrast, the VR market shares for 198 subjects across the 16 brands of breakfast cereal used in the investigation cumulated to 69.3%; the 16 brands of margarine had a combined VR market share (across n = 107) of 52.3%; and the 16 brands of headache remedy had a combined VR market share (across n = 104) of 90.6% (see Table 2). Let us consider the results for each of these products while bearing this point in mind.

TABLE 2

UNADJUSTED MARKET SHARE ESTIMATES

The Pearson r calculated between the BP and VR market share estimates for breakfast cereal was .48. Two factors are relevant to an evaluation of this coefficient. First, due to an oversight, when subjects indicated that they had purchased Raisin Bran during the preceding 12 months, the interviewers had not been instructed to probe with a follow-up question in order to determine just which brand (Kellogg's or Post) was involved. Hence, the VR data for these subjects could not be counted for either of these brands and were therefore omitted from the calculation of the VR market shares. (The BP data suggest that most of these purchasers would have specified Post.) Second, Quaker 100% Natural had recently been introduced into the market area and had not had sufficient time to have been purchased with any great regularity during the preceding year. There is thus reason to believe that the VR estimates of market share for the year were not an accurate reflection of purchase probabilities existing at the time of the investigation. When these 3 problem brands are removed from the set of 16 test brands, the Pearson coefficient increases to .73 (p < .01). With only the two Raisin Bran's removed, the coefficient increases to .64.

The Pearson coefficient for margarine is a highly significant .83. However, a problem exists similar to that for breakfast cereal. If a respondent recalled Blue Bonnet but did not differentiate just which Blue Bonnet was meant, these data could not be used in calculating the VR market share estimates. With the three Blue Bonnet brands omitted, the Pearson Coefficient decreased to .57 (p < .05).

The VR and BP estimates for headache remedies seem to be in fairly high agreement, and a Pearson product moment correlation between the two was highly significant (r = .98). The high (90.6%) percentage of VR market share estimates entering into this correlation probably makes it the most dependable one of the three samples.

It should be acknowledged, however, that the obtained correlations, while moderately high (see Table 3), are based on only 16 (or 13) sets of figures so that changes in 2 or 3 pairs of figures would be sufficient to exert dramatic impact on the coefficients obtained. Second, we are again confronted with the problem that the VR measure assessed behavior over a series of purchase instances whereas the BP indicant was based on only a single instance. Finally, it should also be noted from Table 3 that the cumulated VR market share estimates (when adjusted for confounds) for a given product appear to be positively related to the strength of the relationship manifested between VR and BP estimates -- the greater the amount of VR market share included in our BP test situation, the higher the relationship between the two estimates.

TABLE 3

CORRELATIONS BETWEEN VR-DERIVED AND BP-DERIVED MARKET SHARE ESTIMATES

The NSF investigation also supplied relatively crude data regarding Relationship 2 in Figure 1. Readers familiar with this research from earlier reports (e.g., Jacoby, 1975a; Jacoby, Chestnut, Fisher, and Weigl, 1976; Bettman and Jacoby, 1976) may recall that subjects were provided with "cents-off" coupons which they were later able to apply toward purchases of their test product. Breakfast cereal and margarine subjects received five such "30c-off" coupons, with three of these being good only toward purchases of the brand they had selected at the end of their BP choice task. The other two coupons were good toward the purchase of any brand in that product category, be it the one they had selected in the BP simulation or another. The coupons were valid for a period of four months from the date that the individual participated in the study.

An indication of predictive validity is obtained by addressing the following question: To what extent was the brand selected in the BP task predictive of the brand later purchased (Relationship 2, Figure 1)? Table 4 provides data pertaining to predictive validity for the two margarine conditions. While the vast majority of people redeemed at least one coupon (90% in the BNP sample and 83% in the BNA sample), considerably more coupons were redeemed for the BNP sample (64%) than for the BNA sample (51%). (The data for BNP are comparable to those obtained with breakfast cereal, where 89% of the subjects redeemed at least one coupons and 67% of all coupons were returned.) This pattern of coupon redemptions suggests that the coupons had value for the subjects, and providing five such coupons most probably generated a realistic degree of task motivation, as was intended. Unfortunately, using designated coupons as a manipulation of motivation meant that validity data based on these coupons would be confounded. Moreover, while redemption of the undesignated coupons provides somewhat cleaner data, they are nonetheless affected by the fact that each subject also possessed three designated coupons and these might have influenced how he used his undesignated coupons.

TABLE 4

COUPON REDEMPTIONS FOR MARGARINE

Notwithstanding these problems, Table 4 reveals the following pattern: (1) A higher proportion of coupons were returned by the BNP (as compared to the BNA) subjects, both overall and for the brand they selected in the BP task. This is not surprising since 72% of BNA subjects chose a brand at the end of the BP task which they supposedly didn't purchase (according to an independent VR assessment). The motivation for these subjects to redeem their coupons could be assumed to be low. (2) In contrast, BNA subjects used a higher proportion of their undesignated coupons to redeem unselected brands. (3) Approximately 10% of the undesignated coupons returned by the BNA sample was for their selected brand, while this figure was 40% (24/60) for the BNP sample. The 10% figure for the BNA sample is only slightly higher than the 6.125% that could be expected to have selected their favorite brand in the BP choice task (i.e., there being 1 out of 16 chances of selecting this brand). The 40% figure for the BNP sample is just about what one might have expected given that these subjects already possessed three coupons valid only for their selected brands (and might have therefore chosen one other brand for the sake of variety) and also because the weighted mean VR brand loyalty score for this group was 48 on a 100 point base, indicating that slightly less than half their purchases were devoted to their most often purchased brand in that product category.

RELIABILITY

The key theme of reliability is that of consistency in response. Such consistency can be examined at various levels. As applied according to the traditional approach, an attempt is made to hold virtually everything constant in regard to the assessment except the fact that assessment takes place at two different points in time. That is, the same measuring device (or approach) is applied to the same individuals, in regard to the same stimuli, in the same testing environment, by the same test administrator, etc. However, to be able to generalize, one must also explore consistency in response across different subjects from the universe of all possible subjects of concern, different stimuli from the universe of all possible stimuli of concern, different testers, different environments, etc. That is, according to an increasingly influential contemporary interpretation, traditional reliability assessment expands into the quest for generalizability (see Cronbach, Gleser, Nanda, and Rajaratnam, 1972).

This paper describes data relating to several points along this consistency continuum. First examined is the consistency of response within the same subject across two different points in time with respect to the same decision. Next, we consider the consistency of response within the same subject across qualitatively different decisions. Finally, we consider the consistency of response across different subjects and different decisions. Further, while the section on validity considered both information acquisition and choice behavior, hardly any relevant data have been collected regarding the reliability of choice. Hence, our attention is generally confined to pre-decision information acquisition, particularly the depth (i.e., how much?) and sequence (in what order?) of such information acquisition.

Same Subject, Same Decision, Cross-Time Consistency

One fundamental form of traditional reliability is test-retest reliability -- that is, will the same measurement procedures administered at two different points in time to the same individuals and under the same conditions yield identical (or nearly identical) results? Naturally, this assumes no relevant change during the period intervening between the first and second assessment in the item(s) being assessed. Another basic variety of traditional reliability is parallel form -- that is, to what extent do alternative or equivalent forms of an instrument (or, in this case, a measurement procedure), when applied to the same content, yield comparable results?

Preliminary answers to these questions come from a study in which 71 undergraduates were asked to arrive at a "blind date" choice from among 8 hypothetical options, each described along 15 information dimensions. Approximately half this sample was re-tested after a ten-minute delay during which they were engaged in an interpolated activity; the other half was re-tested approximately one week later. Further, some subjects were given the identical information, with only the labels designating the alternatives being changed (either from letters to numbers or vice verse), while other subjects were confronted with entirely new information for eight different blind dates. The Different condition reflected parallel forms reliability, while the Same condition represented test-retest reliability.

Table 5 indicates the number of subjects in each condition and correlations obtained.

TABLE 5

WITHIN SUBJECT, WITHIN DECISION (TEST-RETEST AND PARALLEL FORM) CONSISTENCY

In general, these coefficients reveal a pattern of high test-retest reliability for the Immediate retest condition, particularly when Different rather than Same stimuli were employed. Also, and as might have been expected, coefficients were generally lower under the Delayed condition. However, they are quite comparable to the test-retest reliabilities obtained for a set of VR items collected for comparison purposes from the these Delayed retest subjects (see Table 6). These latter data suggest that the Delayed/Same subjects might have been somewhat strange. In particular, consider the coefficients for items 2 and 4. Additional data are being collected to clarify this issue.

TABLE 6

CONSISTENCY IN VR RESPONSES FOR SUBJECTS IN THE DELAYED RETEST CONDITION

The two Same cells provide our only indication of the consistency of choice (in contrast to the consistency of information acquisition behavior, which we have been considering up to this point). For any given blind date selected at t1, the probability of selecting this same date at t2 was 1 out of 8 (or .125). Twelve out of 17 (71%) subjects selected the same date on the Immediate retest, while 5 out of 12 (42%) did so for the Delayed retest.

Consistency within the Same Subject across Different Decisions

Thus far, our published reports have considered pre-decision information acquisition for individuals making purchase decisions. However, as suggested by the study above, the BP approach is applicable to other types of decisions as well. One attempt to explore the psychometric properties of the approach involved having the same subjects arrive at three qualitatively different kinds of decisions: purchase one of eight brands of toothpaste; select one of eight (hypothetical) individuals as a blind date; and, choose (vote for) one of the eight candidates on the 1976 U.S. Presidential primary ballot in Indiana. The number of information dimensions for each task was 17, 15, and 6, respectively.

Attention is confined here to the (Brand) Names Absent conditions for the following reasons. First, while the matrix of information used for the toothpaste and political candidate decisions was "real" information (in the sense of actually being available in the environment), this was the only condition possible in the case of the blind dates, where the different alternatives were all hypothetical. Second, when names are present, respondents can rely upon the information they have in memory regarding each of these different options. As a result, we are unable to determine just which information is accessed and considered relevant (since information may also be retrieved from long term memory). Further, since each subject probably had different past experiences with respect to the various toothpastes and candidates, a BNP condition would introduce too many uncontrolled and potentially confounding factors. Finally, when the identity of the choice options is known (as in BNA), a large proportion of subjects (e.g., 36% to 46% in the NSF investigation, see Table 7) do not access any information from the external environment. Thus, (Brand) Names Absent was used to insure that subjects engaged in at least some information acquisition behavior.

TABLE 7

CROSS-SUBJECT, CROSS-DECISION TASK CONSISTENCY

Table 8 provides the relevant data for the 31 subjects on 3 depth and 2 sequence variables. All 15 Pearson coefficients were positive, highly related (median r = .57), and statistically significant -- 14 at beyond the .01 level and 11 of these at beyond the .001 level. Thus, there seems to be a high degree of consistency in the depth and sequence of information acquisition within subjects who arrive at qualitatively different types of decisions.

TABLE 8

WITHIN-SUBJECT, CROSS-DECISION CONSISTENCY

Finally, the NSF investigation (Jacoby and Chestnut, 1977) provides data regarding consistency across heterogeneous decision tasks. The 615 subjects in the 5 samples were interviewed over a 2-year period (1975-76) by more than 12 different interviewers. Respondents reflected a cross-section of Tippecanoe Co. (Indiana) residents above the age of 18 and were representative in terms of age, education, and total family income. All had been "qualified" in advance as users of all three test products.

Table 7 presented the relevant data for six depth, six content, and four sequence variables. The depth variables are fundamental and need no further explanation. The content variables cover the three information dimensions common to all three test products. The sequence variables are those which we developed in 1974 and have been using ever since. Readers are directed to Jacoby, Chestnut, Weigl, and Fisher (1976) for a more detailed description of these indices. Given the many possible configuration of values that the data could assume, inspection of Table 7 reveals an exceptionally high degree of consistency, particularly when one considers the three groups nested under BNP as one comparison set, and the two groups under BNA as another.

DISCUSSION

Appearances (i.e., the many tables of data presented) to the contrary, we have here only touched the tip of the data iceberg. Space limitations do not permit a more exhaustive presentation of data or discussion of findings. Based upon what we have been able to present, however, the tentative findings may be summarized as follows:

1. The convergent validity between VR and BP assessments tend to be in the order of .2 to .4 for information acquisition behavior and range from .48 to .98 for choice behavior. One would expect such greater agreement in regard to a single global act such as choice relative to agreement on the more molecular behavior of information usage. One way to look at this finding is to say that the BP approach has more unique variance to contribute in the study of information acquisition behavior.

2. With respect to reliability, BP measures of information acquisition behavior tend to be quite consistent: (a) across time for different subjects, different interviewers, and different (but comparable) decision tasks; (b) within the same person, across different decision tasks; and (c) within the same person, for the same decision task, across time (i.e., under both immediate and delayed re-test conditions). Further, BP measures seem to be at least as reliable as VR indicants. While we would agree that this assertion is based on little data, the fact of the matter is that not much data have been supplied in the consumer realm to establish the reliability of our traditional VR measures. As examples, based on a comprehensive review of more than 300 studies in the brand loyalty literature, Jacoby and Chestnut (in press) found only two studies which provided reliability data. Similarly, a fairly thorough review of the hundreds of articles comprising the advertising research literature published during the seven years spanning 1968-1974 revealed only two studies bearing on the test-retest reliability of VR measures used in that domain (Jacoby, 1976, p. 6).

Considerable additional evidence bearing on the validity and reliability of BP measures has already been collected and is being supplemented by ongoing research. These data, which include consideration of other types of validity and reliability, will be more thoroughly examined in Jacoby and Chestnut (in preparation). The primary function of this manuscript has been to provide some of the preliminary empirical evidence which suggests that BP measures demonstrate satisfactory validity and reliability and deserve a place in our methodological armamentarium. Obviously, considerably more evidence must be adduced to substantiate this assertion; we are proceeding with this task.

REFERENCES

James R. Bettman, "Information Processing Models of Consumer Behavior," Journal of Marketing Research, 7(August, 1970), 370-376.

James R. Bettman and Jacob Jacoby, "Patterns of Processing in Consumer Information Acquisition," in B. B. Anderson (ed.), Advances in Consumer Research: Volume III. (Cincinnati: Association for Consumer Research, 1976), 315-320.

James R. Bettman and Pradeep Kakkar, "Effects of Information Presentation Format on Consumer Information Acquisition Strategies," Journal of Consumer Research, 3 (4, 1977), 233-240.

Donald T. Campbell and Donald W. Fiske, "Convergent and Discriminant Validation by the Multi-Trait x Multi-Method Matrix," Psychological Bulletin, 5 (1959), 81-105.

Robert W. Chestnut, "Information Acquisition in Life Insurance Policy Selection: Monitoring the Impact of Product Beliefs, Affect toward Agent, and External Memory," unpublished doctoral dissertation, Purdue University (1977).

Robert W. Chestnut and Jacob Jacoby, "Consumer Information Processing: Emerging Theory and Findings," in A. Woodside, P. D. Bennett, and J. N. Sheth (eds.), Foundations of Consumer and Industrial Buying Behavior. (New York: American Elsevier, 1977), 119-133.

Robert W. Chestnut and Jacob Jacoby, "Consumer Decision Making and utilization of Preventative Health Care," presented at TIMS/ORSA session on Marketing Research in Health Care, San Francisco, May, 1977.

Lee J. Cronbach, Goldine Gleser, Harinder Nanda, and Nageswari Rajaratnam, The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles (New York: Wiley, 1972). And Jack Jacoby.

Michael J. Donahue, "Concept Formation: A Behavioral Process Approach," in preparation.

James F. Engel, David T. Kollat, and Roger D. Blackwell, Consumer Behavior, 2nd Ed. (New York: Holt, Rinehart, and Winston, 1973).

Jacob Jacoby, "Perspectives on a Consumer Information Processing Research Program," Communication Research, 2(3, 1975a), 203-215. Reprinted in M. Ray and S. Ward (eds.), Communicating with Consumers: The Information Processing Approach (Beverly Hills, California: Sage, 1976), 13-25.

Jacob Jacoby, "Consumer Information Acquisition Behavior,'' colloquium presented to the Department of Psychology, Tilburg University, The Netherlands, May 30, 1975b.

Jacob Jacoby, "Consumer Research: Telling It Like It Is," in B. B. Anderson (ed.), Advances in Consumer Research: Volume III. (Cincinnati: Association for Consumer Research, 1976), 1-11.

Jacob Jacoby, "The Emerging Behavioral Process Technology in Consumer Decision Making Research," in W. D. Perrault, Jr. (ed.), Advances in Consumer Research: Volume IV. (Atlanta: Association for Consumer Research, 1977), 263-265.

Jacob Jacoby and Robert W. Chestnut, Amount, Type, and Order of Package Information Acquisition in Purchasing Decisions, Final report to the National Science Foundation (GI-43687, June 1977).

Jacob Jacoby and Robert W. Chestnut, Brand Loyalty: Measurement and Management (New York: Wiley, in press).

Jacob Jacoby and Robert W. Chestnut, Information Acquisition in Human Problem Solving: A Behavioral Process Approach, in preparation.

Jacob Jacoby and Robert W. Chestnut, "Study of Likely Impact of Disclosure of Life Insurance Costs on Agent and Consumer Behavior," supported by Federal Trade Commission, Contract No. L0226. in progress.

Jacob Jacoby, Robert W. Chestnut, and William A. Fisher, "Simulating Nondurable Purchase: Individual Differences and Information Acquisition Behavior, Journal of Marketing Research, in press.

Jacob Jacoby, Robert W. Chestnut, and William Silberman, "Consumer Use and Comprehension of Nutrition Information,'' Journal of Consumer Research, 4(2, 1977), 119-128.

Jacob Jacoby, Robert W. Chestnut, Karl C. Weigl, and William A. Fisher, "Pre-Purchase Information Acquisition: Description of a Process Methodology, Research Paradigm, and Pilot Investigation," in B. B. Anderson (ed.), Advances in Consumer Research: Volume III. (Cincinnati: Association for Consumer Research, 1976), 306-314.

Jacob Jacoby, Margarete Hefner, Manfred Sch÷ler, Klaus Grabicke, and Hans RaffTe, "Information Acquisition Behavior in Brand Choice Situations: A Cross-Cultural Extension," Purdue Papers in Consumer Psychology, No. 162, 1976.

Jacob Jacoby, George J. Szybillo, and Jacqueline Busato-Schach, "Information Acquisition Behavior in Brand Choice Situations," Journal of Consumer Research, 3(4, 1977), 209-216.

Carol A. Kohn and Jacob Jacoby, "Operationally Defining the Consumer Innovator," Proceedings, 81st Annual Convention, American Psychological Association, 8(2, 1973), 837-839.

Brenda N. Major and Robert W. Chestnut, "Examining Attribution Processes with an Information Processing Methodology," in preparation.

Jum C. Nunnally, Psychometric Theory (New York: McGraw Hill, 1967).

John W. Payee, "Heuristic Search Processes in Decision Making," in B. B. Anderson (ed.), Advances in Consumer Research, Volume III. (Cincinnati: Association for Consumer Research, 1976), 321-327.

Hans Raffle, Margarete Hefner, Manfred Sch÷ler, Klaus Grabicke, and Jacob Jacoby, Informationsverhalten und Markenwahl," Die Unternehmung, 2(1976), 95-107.

J. Edward Russo and Larry D. Rosen, "An Eye Fixation Analysis of Multi-Alternative Choice," Memory and Cognition, 3(May, 1975), 267-276.

David A. Sheluga and Jacob Jacoby, "Do Comparative Claims Encourage Comparison Shopping? -- The Impact of Comparative Claims on Consumers' Acquisition of Product Information," in J. Leigh and C. R. Martin (eds.), Current Issues and Research in Advertising, in press.

Time magazine, March 22, 1976.

W. Fred van Raaij, Consumer Choice Behavior: An Information Processing Approach (Voorschooten, The Netherlands: VAM, 1977a).

W. Fred van Raaij, "Consumer Information Processing for Different Information Structures and Formats," in W. D. Perrault (ed.), Advances in Consumer Research: Volume IV (Atlanta: Association for Consumer Research, 1977b), 176-184.

----------------------------------------