The Stability of Responses Obtained By Free Elicitation: Implications For Measuring Attribute Salience and Memory Structure

ABSTRACT - This paper examines the stability over time of consumers' verbal responses in a variety of free elicitation tasks and the test-retest reliability of several measures derived from those responses. The paper concludes with a discussion of the potential usefulness of the free elicitation procedure for identifying salient product attributes and for indicating the content and organization of product knowledge structures as stored in memory.

Citation:

Jerry C. Olson and Aydin Muderrisoglu (1979) ,"The Stability of Responses Obtained By Free Elicitation: Implications For Measuring Attribute Salience and Memory Structure", in NA - Advances in Consumer Research Volume 06, eds. William L. Wilkie, Ann Abor, MI : Association for Consumer Research, Pages: 269-275.

Advances in Consumer Research Volume 6, 1979      Pages 269-275

THE STABILITY OF RESPONSES OBTAINED BY FREE ELICITATION: IMPLICATIONS FOR MEASURING ATTRIBUTE SALIENCE AND MEMORY STRUCTURE

Jerry C. Olson, Pennsylvania State University

Aydin Muderrisoglu (student), Pennsylvania State University

[This research was funded by a Research Initiation Grant to the first author from Pennsylvania State University. The authors are grateful to Rajesh Kanwar and Sik Shum for their assistance in data collection.]

ABSTRACT -

This paper examines the stability over time of consumers' verbal responses in a variety of free elicitation tasks and the test-retest reliability of several measures derived from those responses. The paper concludes with a discussion of the potential usefulness of the free elicitation procedure for identifying salient product attributes and for indicating the content and organization of product knowledge structures as stored in memory.

INTRODUCTION

A variety of marketing research issues involve determining the content of consumers' cognitive structures for products, brands, or other concepts. A common example is the need to identify the product attributes consumers believe to be salient (Sampson and Harris, 1970), important (Cohen, Fishbein, and Ahtola, 1972), or determinant (Myers and Alpert, 1968; 1977). Once identified, these product characteristics are typically incorporated into a structured questionnaire. Consumer responses are then obtained in a mass administration and are usually combined via some multi-attribute model to predict consumer attitudes or choice behavior (cf. Day, 1972; Wilkie and Pessemier, 1973). A second example involves a related, but more basic issue--namely, how consumers encode, organize, and store in memory their knowledge about a product and its characteristics (Johnson and Russo, 1978; Olson, in press). Determining the content and organization of a consumer's product knowledge structure requires procedures and techniques by which a researcher can identify (a) the cognitive form or symbol the consumer uses to represent a product characteristic, (b) the meaning (comprehension) "attached to" that symbol, (c) the relative "importance" (vis-a-vis some criterion) of that knowledge representation, and (d) the interrelationships or structure among various product knowledge characteristics. This paper describes one such procedure--free elicitation--and explores the stability of the data generated by it.

A number of approaches and techniques have been used in attempts to identify the salient or "important" dimensions of cognitive structure. Perhaps most commonly reported in the marketing literature is the use of a loosely-structured individual or group interview, the details of which unfortunately, are seldom described. More structured and therefore more replicable procedures exist, but these have been used much less frequently in marketing research. Examples include the Kelly (1955) repertory grid (Frost and Baine, 1967; Sampson, 1972) or modifications thereof (Wilson and Dover, 1975), cued or "controlled" elicitations (Fishbein, 1967), and free recall (Green, Wind and Jain, 1973; Johnson and Russo, 1978). An exhaustive review of the pros and cons of these techniques is beyond the scope of this paper, as is a detailed rationale for the selection of the elicitation procedure advocated here Suffice it to say that recent developments (Abelson, 1973, 1976; Calder, 1978) indicate that peoples' cognitive structures contain a variety of types of concepts including abstract, attribute-specific dimensions as well as more concrete, visual concepts such as images (Mower, 1977) and scripts (Shank and Abelson, 1977). Thus, it was deemed important that the procedure to be investigated was sufficiently "open" and nondirective that the full variety of cognitive elements maintained in a memory structure could be evidenced. This paper examines a free elicitation procedure in which respondents are free to say anything and everything that comes to mind when presented with a stimulus probe cue.

In cognitive research, a common method for investigating the content and organization of memory structure is to ask the respondent to freely recall previously learned information (cf. Buschke, 1977). The free elicitation procedure of interest here is essentially the same, but for two important differences. First, in free recall the probe cue that triggers responding is a phrase such as, "Tell me the words we learned yesterday..(or, last week)." In free elicitation, the probe tends to be more general and non-time specific--an attempt to "trigger" or activate a particular structure of stored knowledge--for example, "Tell me what comes to mind when I say Ford LDT!" Second, in contrast to free recall, the particular learning event or the content acquired from a specific learning experience tends not to be of interest in free elicitation Rather, the researcher is primarily interested in the content and organization of an existing structure of knowledge stored in semantic memory (Olson, in press).

Once generated, the responses elicited in either free recall or free elicitation tasks can then be examined as evidence of memory content and organization. A wide variety of measures and indices can be developed from free response data. For example, one could examine the ordering of the elicited concepts, the clustering patterns of concepts, the rate of elicitation, the time taken between responses, etc. Moreover, one can examine the nature of individual responses. For example, is the elicited concept an abstract product characteristic such as convenience, or is it more concrete (i.e., more visual or imaginal)? Although free response procedures have been used occasionally in cognitive research, there seems to be little published evidence regarding the stability of such data, or the reliability of measures derived from such data, or, indeed, the validity of the approach for indicating the content and organizational characteristics of knowledge structures. Thus, before the free elicitation procedure can be proposed as possibly useful for indicating consumers' memory content and structure, the psychometric qualities of the data need to be established, beginning with issues of stability and reliability.

A variety of approaches to establishing reliability have been identified (cf. Nunnally, 1967; Silk, 1977) and each has specific advantages and particularly appropriate applications (Peter, 1977). Underlying all theories of reliability, however, is the idea of "consistency" or "stability" of response, and this is the focus of interest here. The present study involves a variety of single measures or indices (of possibly different constructs), most of which are not yet fully explicated. Thus, the reliability approach appropriate for this somewhat primitive state of measurement must focus on the test-retest reliability of individual measures. The hope, of course, is that the free elicitation procedure is capable of producing reasonably stable sets of responses and that important aspects of these responses can be described by reliable measures. If so, subsequent attention can be devoted to developing theoretically meaningful interpretations of the measured constructs in terms of memory, content and organization.

METHOD

Research Overview

Naturally occurring cognitive structures were the focus of this study. That is, there were no experimental manipulations of the product knowledge that consumers had stored in memory. To produce additional data per subject and to allow examination of differential effects of factors such as product familiarity and ego-involvement, free responses were elicited vis-a-vis three product categories--tooth-paste, ball-point pens, and blue jeans. [These three products were selected essentially arbitrarily to meet the criteria of common purchase and use by a student population.] Because the three products examined were common and frequently purchased by the subject population, subjects were presumed to have previously acquired rather stable knowledge structures for each of the product categories, derived from their fairly extensive past experience.

The knowledge structures associated with each product were examined by presenting subjects with three different types of stimulus cues (memory probes). The "broadest" probe cue was the product category name (e.g., "toothpaste"). A somewhat more specific probe involved presenting subjects with a combined product and purchase situation stimulus. The third type of probe was a specific brand name. Consumers were instructed to state "all the things that come to mind" each time the researcher mentioned a probe word or phrase. In order to examine the reliability of the measures of the concepts generated in these free elicitation tasks and the stability of the concepts them- selves, consumers returned after one week and repeated the elicitations.

Subjects

Thirty male and female undergraduate business students enrolled in the introductory marketing course volunteered to participate in the study in return for a $4.00 cash payment. [Clearly, the external validity of this sample is limited. Rather, the purpose of this study is to demonstrate the potential stability of free elicitation data in the context of a particular subject population, not to demonstrate its generalizability to a wider population.] Subjects were run individually, and each free response session required about 45 to 50 minutes with a subject.

Elicitation Procedures

After a subject entered the study room, he/she was directed to a seat across the table from the researcher. On the table was a cassette tape recorder. After introducing himself and stating that the study concerned the subject's reactions to several common products and brands, the researcher handed the subject a written copy of the instructions for the first part of the study (see Appendix). To insure that each subject acquired a clear and thorough understanding of the free response procedures, the subject was asked to follow along on his copy while the researcher read the instructions aloud, verbally emphasizing important points (critical parts of the instructions are underlined in Appendix). In essence, the instructions asked the subject to verbally state, as rapidly as possible, [The speeded aspect to the elicitation procedure is critical. The notion is that rapid responses are less subject to biases (e.g., social desirability and intentional distortion, and thus reflect more accurately the content of memory.] any and all thoughts that came to his or her mind when the experimenter said the name of a product or brand. Subjects were told that it was too difficult to write down the responses and that, for accuracy, the responses would be recorded (but kept anonymous.

Then the recorder was turned on and the researcher said, "OK; tell me the things that come to mind when I say. . soft drinks." This first response elicitation constituted a "warm-up" task for the subject to become familiar with the procedure and these responses were not analyzed. If the subject was too verbal or analytical in describing or justifying single thoughts (an occasional problem), the researcher pointed out that short phrases descriptive of each thought were sufficient. Then the researcher began the elicitations of interest by stating the name of one of the three products of particular concern--ballpoint pen, toothpaste or blue jeans. After the subject had stated his elicited thoughts for the first product, the researcher stated the names of the second and the third products and tape recorded the elicitations produced for each. The order of the three products was randomly determined for each subject. After each subject had freely responded to the three general product categories, the researcher stopped the recorder and told the subject to complete one of three two-page questionnaires (one for each product category, randomly selected). Each instrument contained questions about product usage, self-confidence, product importance, etc.

Subjects were then given the instructions for the second part of the study (see Appendix). Here, subjects were asked to verbally state the concepts and thoughts that came to mind when considering buying a product for their personal use, when told the product's name. The recorder was turned on and the researcher stated a product/personal use purchase situation cue for the same three products as before, one at a time in random order, and the subject's freely elicited responses were recorded for each. Then the recorder was stopped and the subject completed in random order the two-page questionnaires for the remaining two products.

Then, the subject was handed the instructions for the third' part of the study (see Appendix) and was asked to follow them as they were read aloud by the researcher. The elicitation procedure was similar to that used previously except that the subject was to respond to specific brand names. Two well-known brand names were selected for each of the three products--Crest and Ultrabrite, Bic and Cross, Levi's and Wrangler. [These brand names were selected arbitrarily except for the criterion that they be familiar to most subjects.] Subjects were to verbally state the thoughts that come to mind when presented with each of these six brand names.

Following the elicitations, a sorting procedure was used to measure subject's ego-involvement with the three products and attitude and purchase intention measures were taken. First, each subject was given one of three decks of 5 x 7 index cards (chosen at random). On the cards in each deck were printed all of the locally-available brand names for that product. The subject was asked to sort through the card deck and remove those brands he/she had not heard of before. Then, the subject was told to examine the remaining brands/cards and sort them into three piles or categories, corresponding to the latitudes of acceptance, rejection, and neutrality (Sherif, Sherif and Nebergall, 1965). [Due to space limitations, these data are not presented here.] Subjects then estimated the percentage of their product purchases over the past year that were devoted to each brand in the acceptance region. Finally, subjects indicated their attitude ("_o--5-point bad-good scale) and purchase intention (BI---7-point very unlikely-very likely scale) toward each familiar brand. After subjects finished the sorting and rating tasks with one product, the same procedures were repeated for the other two products, again in random order.

At the conclusion of the first session, subjects were given one dollar and told to return to the same room at the same time and day exactly one week later for the conclusion of the study. Subjects were advised of the importance of attending the second session and were reminded that they would receive the remaining $3.00 of payment at that time. When each subject arrived for the second session, the study procedures were repeated exactly, with the same products and brands, except that the order of products and brands within each stage of the replication was randomly determined.

RESULTS AND DISCUSSIONS

The general purpose of this study was to determine the stability of freely elicited concepts and the reliability of summary measures of the concepts. In the context of this study, stability means that a probe cue (a word spoken by the researcher) presented to consumers at two points in time elicited similar responses. The reliability of measures of those responses is indicated by test-retest correlation coefficients. The results are discussed separately for the elicitations produced by each type of probe cue.

Product Category Probes

Table 1 presents the means and reliabilities for selected measures of the concepts elicited in response to the product name cues. The total number of concepts elicited for the toothpaste and ballpoint pen probes was reasonably reliable (.77 and .74), but was less so (.47) for blue jeans. Time of response yielded only moderate reliabilities (r's = .43 to .50), as did the rate of elicitation (.21 to .73). The degree to which similar concepts were elicited in the test-retest tasks was about the same for all three products (47% to 52%), and the rank order correspondence of these similar concepts over the two elicitations was positive but weak (.09 to .23). For all three product categories, abstract product attributes constituted approximately 70% of the similar elicited concepts, while the remaining 30% were more concrete. [Elicited concepts were judged as concrete if they described a visual image or a sequence of images (script), or expressed a probable action, or referred to particular appropriate usage occasions. Abstract concepts were those relating to product attributes or characteristics (see Geistfeld et al., 1977).] The rank order correspondence among the abstract concepts was weakly positive (.20 to .29).

Product/Purchase Context Probes

The stability/reliability measures for the concepts elicited in response to the combined product/purchase context probes are presented in Table 2. The reliabilities of the number of concepts elicited were reasonably strong (.54 to .77) and the reliabilities for time taken and rate of elicitation were moderate (.35 to .54 and .32 to .64). The degree to which the same concepts were elicited in the two trials was about the same for the three products (54% to 60%), and the rank order correspondence of the similar elicited concepts was positive and weak (.22 to .34). Again, abstract product attributes accounted for approximately 70% of the similar elicited concepts and the elicitation rank order correspondence between these product dimensions was moderate (.28 to .45).

TABLE 1

MEANS AND RELIABILITIES FOR SELECTED MEASURES OF THE STABILITY OF FREE ELICITATIONS TO PRODUCT CATEGORY PROBES

TABLE 2

MEANS AND RELIABILITIES FOR SELECTED MEASURES OF THE STABILITY OF FREE ELICITATIONS TO PRODUCT/PURCHASE SITUATION PROBES

TABLE 3

MEANS AND RELIABILITIES FOR SELECTED MEASURES OF THE STABILITY OF FREE ELICITATIONS TO BRAND NAME PROBES

Brand Name Probes

Table 3 presents the reliabilities for selected measures of the concepts elicited in response to the six brand name probes. Much the same pattern of stability and reliability was obtained as for the other probe types. The number of concepts elicited and the time and rate of elicitations were moderately reliable over the test-retest tasks. Approximately 60% of the concepts elicited at T₂ were also elicited at T₁, and the rank order correspondence of elicited concepts was moderate (.24 to .41). Abstract product attributes constituted approximately 80% of the similar elicited concepts and the elicitation rank order correspondence among these abstract dimensions was moderately positive (.29 to .48).

Summary and Discussion

The general pattern of results indicates that the free elicitation procedure is capable of producing responses which are moderately stable. Moreover, measures of these responses appear to be reasonably reliable. Although reliabilities of .85 to .95 might have been hoped for, such high levels of stability/reliability are unlikely for such data. The authors believe, in fact, that the levels of stability and reliability evidenced here are quite encouraging, given the nature of the elicitation task. Free elicitation responses may be influenced by a number of variables in addition to the presumed major causal factor--namely, the underlying cognitive knowledge structure in memory. Considered in this light, the obtained reliabilities and indices of stability seem acceptably strong.

Another perspective on the magnitude of the obtained reliability indices may be gained by examining the test-retest reliability coefficients obtained for variables more commonly measured in marketing research. Reliabilities for single measures of attitude and purchase intention toward each of the six brands ranged from .65 to .89. Subjects' self-reports of their past brand purchasing behavior were also quite reliable (.60 to .92). The magnitude of these reliability coefficients suggest the likely levels of reliability that can be obtained for relatively simple, direct, almost standardized measures of presumably stable constructs. Compared to these somewhat stronger levels, the reliabilities obtained for the free elicitation procedure appears "about right" and encouragingly positive, given the nature of the method and the "freedom" of response.

CONCLUSIONS AND IMPLICATIONS

The present results indicate that the free elicitation procedure can generate concepts that are moderately stable over repeated elicitations. Moreover, those concept responses can be described by measures that possess encouraging levels of reliability. Before examining the implications of these data, however, their limitations should be reviewed.

Limitations

A major limitation to any test-retest study is the possibility of alternative explanations for the reliability coefficients (Silk, 1977). Like most research, this investigation unavoidably confounds the reliability of of the empirical measure and the stability of the measured construct. [A procedure does exist which can separate the reliability of the measure from the stability of the construct being measured (Heise, 1969). However, the measure must be taken three times under a set of rather restrictive assumptions.

Thus, it is not possible to know whether, for example, a test-retest correlation of .7 means that the measure of a stable construct is moderately reliable, or that a moderately stable construct is measured by a totally reliable measure, or that the measure of a somewhat unstable construct is somewhat unreliable. In the present study, the first elicitation may have had some effects (undefined at this point) on the underlying knowledge structure in memory. If so, the present reliability coefficients reflect a certain degree of non-stability or change in the underlying construct. However, the present study did conduct the retest procedure under conditions nearly identical to the first elicitation, and therefore the possible influences on the stability of the construct were minimized. Another limitation involves the restricted external validity of these results to other subject populations, product categories, and brands. However, on purely logical grounds it would seem that the free elicitation procedure should produce equivalent levels of reliability/stability for adult consumers who have levels of verbal fluency similar to college students.

In general, the present results indicate that the free elicitation procedure can produce sufficiently stable data and reliable measures that further research with the technique is warranted. The procedure has particular implications for two issues of importance in consumer research, attribute salience and memory structure.

Attribute Salience

Because of their concern with how consumers perceive and choose among alternative products and brands, consumer researchers are especially interested in the specific attributes or product characteristics that consumers consider salient--i.e., as "important" in some sense. Although various methods for establishing attribute salience have been used (Myers and Alpert, 1977; Sampson and Harris, 1970; Wilkie and Pessemier, 1973), the basic procedure advocated by Fishbein (1967) is an elicitation procedure. Fishbein's typical approach, however, has not been to use completely free elicitation. Rather he has given subjects rather specific instructions regarding the type of concepts desired, e.g., "tell me what you believe to be the characteristics, qualities, and attributes of. . ." (Fishbein and Ajzen, 1975; Kaplan and Fishbein, 1969). Marketers using elicitation procedures also appear to ask consumers to state "the product characteristics that come to mind. . .". Because of this emphasis on product characteristics, it is not surprising that the typical multi-attribute model questionnaire used in marketing research includes belief ratings only about relatively abstract product attributes.

It is becoming recognized, however, that other types of cognitive representations may be related to attitudes (cf. Abelson, 1976; Calder, 1978). These include concrete visual images and scripts (Shank and Abelson, 1977) as well as relatively concrete attributes (e.g., color; see Geistfeld, Sproles, and Bradenhop, 1977). Only a totally unrestricted type of elicitation task as used in this research is likely to generate these types of concepts. The present study consistently found that 20 to 30% of the stable concepts elicited on both trials were more concrete image or script type responses, while the remaining 70-80% were the more abstract product attributes. As a brief example, a common script type elicitation for the toothpaste brand probes was the description of a scene or scenes from a familiar television commercial. Attitude theorists and applied marketing researchers should begin to consider such cognitive elements and attempt to determine their effects on attitude formation and choice behavior.

Fishbein's operationalization of salience has relied heavily on the ordering of elicited concepts, with the first few (5 to 9) supposedly most likely to be salient (or most salient). The data reported here suggest caution in interpreting elicitation orderings in this manner. The test-retest correspondence of the elicitation rank orders were uniformly positive but not extremely strong (all tau's < .50). [Interpretation of the low positive rank order correlations must be tempered by the recognition that when the number of concepts is few, as in the present case (3 or 4 similar concepts was typical), a single reversal in rank order dramatically lowers the tau coefficient.] If one cannot be confident of the stability of an ordering of elicited concepts, then one should not be confident in a theoretical interpretation of that order. Of course, replications using other subject samples and other products are required to determine the "stability" of this finding.

Finally, the present study found that roughly 50 to 60% of the concepts elicited in a second trial were also elicited in a previous trial. Therefore, marketers may wish to restrict their definition of salience to only those concepts that are reliably elicited over multiple probings, and not be particularly concerned about the less stable rank ordering of those similar concepts. Finally, it is interesting to note that the number of similar concepts elicited in both trials is substantially fewer (3 or 4) than the number of salient attributes typically discussed in the marketing and psychological literature (7-9 or even more; cf. Fishbein and Ajzen, 1975; Wilkie and Pessemier, 1973).

Memory Structure

The present results should encourage basic research into semantic knowledge structures stored in memory (Calder, 1975; Olson, in press). Data and measures derived from the free elicitation procedure which might be interpreted as indicating aspects of memory structure (e.g., the number of concepts and the time and rate of elicitation) appear to be at least moderately stable and/or reliable over time. The content of memory structure is also of interest and appears to be measurable with at least moderate reliability by the free elicitation procedure. In response to a variety of probes, a consistent 50-60% of the concepts elicited at time 2 were also elicited at time 1. In sum, free elicitation with its flexibility in terms of easy experimental manipulation of probe cues, may produce interesting insights into how consumers store and organize product/brand information.

Future Research

In further research with the free elicitation procedure, several issues are important. Although, to meet space constraints, the present paper did not examine the effects of individual differences factors such as product ego-involvement, it seems likely that consumers who are more highly involved with a product or brand should possess more well-developed, stable knowledge structures (Olson and Dover, 1978). Thus, such consumers should produce more stable responses (and more reliable measures) in a free elicitation task than those lower in ego-involvement. The effects of individual differences on the stability of free elicitation responses warrant research attention.

A major task for future research is to establish the construct validity of the elicited responses and the various measures thereof. Empirical evidence must be obtained that can support the interpretation of these measures as indicating the content and organization of knowledge structures in a consumer's memory. One possible approach would be to experimentally modify a memory structure, perhaps by exposing subjects to a strongly persuasive message, and then check for evidence of cognitive change by conducting a subsequent elicitation of the contents of the presumably altered structure.

Finally, there is a need to work toward further refine-refinement of the free elicitation procedure itself. For instance, subtle modifications in procedure may cause substantial changes in response stability. To facilitate such explorations and extensions, the present methods have been reported in complete detail.

APPENDIX

Instructions: Part I

We are interested in your thoughts and feeling about several common consumer products. Most of the products are ones that you have bought and probably buy fairly regularly. In the first part of our study, we are interested in your general thoughts about four products. I will tell you the name of each product, one at a time. Then, I want you to simply tell me the things that come to your mind about that product. You should concentrate on the name of the product and verbally tell me your thoughts as completely as possible. You will have about a minute-and-a-half to respond to each product. However, the time limit is not strict, so if you feel you have a lot to say, you may take more time. For some products, lots of things might come to mind. For others, perhaps only a few things will come to mind. You should stop whenever no other thoughts about the product come to mind, even if this happens before the minute-and-a-half is over. But, please tell me all the thoughts you have about each product. In giving your responses, you may have to pause to form your thoughts. Don't let that pause bother you. It is only natural that you might need a few seconds for certain thoughts. But when you do think of something, please state it right away. Please do not censor your comments. We are interested in what you think. If something comes to mind, please say it--even if you might consider it unimportant, ordinary, or even silly. In giving your thoughts, please state each one in short phrase or a word or two. If you feel that you must use more words to express yourself completely or clearly, please feel free to do so. However, we are not interested in why you think your responses are important. So, please do not give justifications or elaborate descriptions of your thoughts--just a simple statement of each thought is sufficient. You should try to give your thoughts as rapidly as possible. Try not to let me distract you in any way. I am just here to give you the instructions and the product names. You might try, either psychologically, or by physically turning in a different direction, or by closing your eyes, to disregard me. I will be recording both our comments during this session on this tape recorder. I am taping the sessions only to be sure I get an accurate record of your responses. However, your name will never be associated with the recording or your specific responses and, when we are finished with the study, the tapes will be erased. Before we start, let me summarize the procedure again. I will first tell you the name of the product. Then you tell me all the things that come to your mind about this product, as rapidly and completely as possible. Do you have any questions before we begin?

Instructions: Part II

In this part of the study, we are interested in your thoughts about each of these products when you are considering buying the product in a specific situation. First, I will explain to you the buying situation of interest. For example, you might assume that you are thinking of buying this product for a specific purpose or occasion. Then, I will tell you the name of the product. I want you to tell me the things that come to your mind about buying that product in that situation. You should try to concentrate on the product and the situation, together, and give me your thoughts as rapidly and completely as possible, just as before.

Instructions: Part III

In this part of the study we are interested in your thoughts about particular brands of certain products. First, I will tell you the name of a brand of one of the products we have been concerned with today. Then, you tell me the things that come to your mind about that brand. Concentrate on the brand and give me your thoughts as rapidly and completely as possible, just as before.

REFERENCES

Robert P. Abelson, "The Structure of Belief Systems," in R. C. Schank and K. Colby, eds., Computer Models of Thought and Language (San Francisco: W. H. Freeman, 1973)

Robert P. Abelson, "Script Processing in Attitude Formation and Decision Making," in J. Carroll and J. Payne, eds., Cognition and Social Behavior (Hillsdale, NJ: Lawrence Erlbaum, 1976)

Herman Buschke, "Two-Dimensional Recall: Immediate Identification of Clusters in Episodic and Semantic Memory," Journal of Verbal Learning and Verbal Behavior, 16 (April, 1977), 201-15.

Bobby J. Calder, "The Cognitive Foundations of Attitudes: Some Implications for Multi-Attribute Models," in M. J. Schlinger, ed., Advances in Consumer Research, Vol. 2 (Chicago: Association for Consumer Research, 1975)

Bobby J. Calder, "Cognitive Response, Imagery, and Scripts: What is the Cognitive Basis of Attitude?" in H. Keith Hunt, ed., Advances in Consumer Research, Vol. 5 (Ann Arbor, Mich.: Association for Consumer Research, 1978)

Joel B. Cohen, Martin Fishbein and Olli T. Ahtola, "The Nature and Uses of Expectancy-Value Models in Consumer Attitude Research," Journal of Marketing Research, 9 (November, 1972), 456-60.

George S. Day, "Evaluating Models of Attitude Structure," Journal of Marketing Research, 9 (August, 1972), 279-86.

Martin Fishbein, "A Behavior Theory Approach to the Relationship Between Beliefs About an Object and the Attitude Toward the Object," in Martin Fishbein, ed., Readings in Attitude Theory and Measurement (New York: Wiley, 1967)

Martin Fishbein and Icek Ajzen, Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research (Reading, Mass.: Addison-Wesley, 1975)

W. A. K. Frost and R. L. Baine, "The Application of the Repertory Grid to Problems in Marketing Research," Commentary, 9 (October, 1967), 161-75.

Loren V. Geistfeld, George B. Sproles and Suzanne B. Bradenhop, "The Concept and Measurement of a Hierarchy of Product Characteristics," in W. D. Perreault, Jr., ed., Advances in Consumer Research, Vol. 4 (Atlanta: Association for Consumer Research, 1977)

Paul E. Green, Yoram Wind and Arun K. Jain, "Analyzing Free Response Data in Marketing Research," Journal of Marketing Research, 19 (February, 1973), 45-52.

David R. Heise, "Separating Reliability and Stability in Test-Retest Correlation," American Sociological Review, 34 (February, 1969), 93-101.

Eric J. Johnson and J. Edward Russo, "The Organization of Product Information in Memory Identified by Recall Times," in H. Keith Hunt, ed., Advances in Consumer Research, Vol. 5 (Ann Arbor, Mich.: Association for Consumer Research, 1978).

Kalman J. Kaplan and Martin Fishbein, "The Sources of Beliefs, Their Saliency, and Prediction of Attitude," Journal of Social Psychology, 78 (June, 1969), 63-74.

George A. Kelly, The Psychology of Personal Constructs (New York: Norton, 1955).

O. Hobart Mower, "Mental Imagery: An Indispensable Psychological Concept," Journal of Mental Imagery, 1 (Fall, 1977), 303-26.

James H. Myers and Mark I. Alpert, "Determinant Buying Attributes: Meaning and Measurement," Journal of Marketing, 32 (October, 1968), 13-20.

James H. Myers and Mark I. Alpert, "Semantic Confusion in Attitude Research: Salience vs. Importance vs. Determinance," in W. D. Perreault, Jr., ed., Advances in Consumer Research, Vol. 4 (Atlanta: Association for Consumer Research, 1977)

Jum D. Nunnally, Psychometric Theory (New York: McGraw-Hill, 1967)

Jerry C. Olson, "Theories of Information Encoding and Storage: Implications for Consumer Research," in Andrew A. Mitchell, ed., Effects of Information on Consumer and Marketing Behavior (Chicago: American Marketing Association, 1978, in press)

Jerry C. Olson and Philip A. Dover, "Attitude Maturation: Changes in Related Belief Structures Over Time," in H. Keith Hunt, ed., Advances in Consumer Research, Vol. 5 (Ann Arbor, Mich.: Association for Consumer Research, 1978)

J. Paul Peter, "Reliability: A Review of Psychometric Basics and Recent Marketing Practices," Unpublished paper, Washington University, 1977.

Peter Sampson, "Using the Repertory Grid Test," Journal of Marketing Research, 9 (February, 1972), 78-81.

Peter Sampson and Paul Harris, "A User's Guide to Fishbein," Journal of the Market Research Society, 12 (July, 1970), 145-89.

Roger Schank and Robert Abelson, Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures (Hillsdale, NJ: Lawrence Erlbaum, 1977)

Carolyn W. Sherif, Muzafer Sherif and R. E. Nebergall, Attitude and Attitude Change: The Social Judgment Involvement Approach (Philadelphia, Pa.: W. B. Saunders, 1965)

Alvin J. Silk, "Test-Retest Correlations and the Reliability of Copy Testing," Journal of Marketing Research, 14 (November, 1977), 476-86.

William L. Wilkie and Edgar A. Pessemier, "Issues in Marketing's Use of Multi-Attribute Attitude Models," Journal of Marketing Research, 10 (November, 1973), 428-41.

David T. Wilson and Philip A. Dover, "A Test of the Repertory Grid Technique as a Means of Developing Attributes for Use in Choice Models." Paper No. 34, Working Series in Marketing Research, College of Business Administration, Pennsylvania State University, 1975

----------------------------------------