Verbal Protocols and Direct Observation of Supermarket Shopping Behavior: Some Findings and a Discussion of Methods

ABSTRACT - A prime characteristic of the information processing approach to the study of behavior is the use of methods that are capable of providing detailed observations of the course of behavior over time (Simon, 1976). Examples of such process tracing techniques are: the collection of verbal protocols, monitoring information acquisition, and response time recording. The purpose of this paper is to discuss some of the benefits and some of the problems associated with the use of one process tracing technique--verbal protocols, to investigate consumer decision behavior. In particular, the use of verbal protocols to study consumer decision making in supermarkets will be illustrated.

Citation:

John W. Payne and E. K. Easton Ragsdale (1978) ,"Verbal Protocols and Direct Observation of Supermarket Shopping Behavior: Some Findings and a Discussion of Methods", in NA - Advances in Consumer Research Volume 05, eds. Kent Hunt, Ann Abor, MI : Association for Consumer Research, Pages: 571-577.

Advances in Consumer Research Volume 5, 1978      Pages 571-577

VERBAL PROTOCOLS AND DIRECT OBSERVATION OF SUPERMARKET SHOPPING BEHAVIOR: SOME FINDINGS AND A DISCUSSION OF METHODS

John W. Payne, Duke University

E. K. Easton Ragsdale, University of Chicago

ABSTRACT -

A prime characteristic of the information processing approach to the study of behavior is the use of methods that are capable of providing detailed observations of the course of behavior over time (Simon, 1976). Examples of such process tracing techniques are: the collection of verbal protocols, monitoring information acquisition, and response time recording. The purpose of this paper is to discuss some of the benefits and some of the problems associated with the use of one process tracing technique--verbal protocols, to investigate consumer decision behavior. In particular, the use of verbal protocols to study consumer decision making in supermarkets will be illustrated.

VERBAL PROTOCOLS

The collection of verbal protocols is conceptually a straightforward method of obtaining process data. The subject is simply asked to give continuous verbal reports, "to think aloud," while performing the task of interest. The verbal protocol is then treated as a "record of the subject's ongoing behavior, and an utterance at time t is taken to indicate knowledge or operation at time t" (Newell & Simon, 1972).

Distinguishing Protocols from Other Verbal Data

The collection of verbal reports is an old idea in psychology. It is also an idea which has generated considerable controversy. Because there is an apparent parallel between verbal protocols and other types of verbal data, I want to stress a few of the distinctive features of the process tracing technique of verbal protocols. First, unlike the earliest introspective techniques, subjects under the process tracing strategy are naive about the theoretical constructs of interest to the researcher. Second, in collecting verbal protocol data the subject is not asked to theorize about his behavior. Instead the subject is asked "only to report the information and intentions that are within his current sphere of conscious awareness" (Newell & Simon, 1972). The researcher, not the subject, does all the theorizing about the cause and consequences of the subject's knowledge state. Third, unlike the traditional way in which verbal reports have been collected--after the response, verbal protocols are collected during the actual performance of the task. While answers to the questions, "What are you doing right now?" may be misleading, they are likely to be considerably less misleading than later descriptions of how the task was performed. Later descriptions allow much more opportunity for the subject to mix current knowledge with past knowledge, making reliable references from verbal reports difficult.

This last distinction is crucial, for the controversy of what subjects can verbally report has reemerged many times. Nisbett and Wilson (1977) recently argued that people have little or no ability to directly observe and verbally report upon higher order mental operations that result in a response of some kind. While the research they review is impressive in supporting their argument, the fact that the evidence is derived entirely from verbal reports collected after the response limits its relevance for evaluating the process tracing technique of verbal protocols. On the other hand, the evidence from years of research on human problem solving behavior does indicate that verbal protocol data can serve as a valuable basis for model building. Obviously, more research is needed to determine the levels and amounts of information about cognitive processes that can be provided by verbal protocols collected in various task environments.

Using Protocol Data

Once the verbal protocols have been collected, they can be used in a variety of methods. For example, verbal protocols may be a particularly valuable source of data during the early phases of an investigation of consumer behavior. The data can be used to search for regularities in behavior. In addition, a preliminary coding of the protocols may serve as the basis for the calculation of descriptive statistics relevant to theoretical issues. Svenson (Note 1), for example, had six subjects make hypothetical decisions to buy one of seven houses presented in great detail in booklets, one for each house. This study took place in a controlled laboratory environment. Information was obtained on the attributes being used to judge the houses. Information was also obtained on the evaluation of a particular house on a particular attribute in terms of a comparison with a criterion or in terms of a comparison with another alternative. Finally, Svenson was able to track possible changes in decision rules over time. The study of supermarket shopping behavior to be presented later represents an example of the use of verbal protocols to do exploratory consumer research in field settings.

Another very valuable use of verbal protocols is to confirm and extend the interpretations of data collected by other methods. In particular, verbal protocols have already been used successfully to support the interpretation of process data collected by the monitoring of information acquisition behavior (Payne, 1976). In addition, Russo and Dosher (Note 2) have used a "prompted protocols" for the same purpose. Prompted protocols are verbal reports given by the subject after the decision has been made but prompted by presenting the subject with a record of his behavior during the task.

Verbal protocols can also be used to test specific hypotheses about behavior in experimental settings. An example of this use of verbal protocols is provided by Montgomery (1976). He was interested in replicating Tversky's (1969) study on intransitive preference patterns. However, unlike Tversky's procedure, the subjects in Montgomery's study were asked to "think aloud" while making their choices. Statements were coded with the focus on explicit dimensional comparisons such as "fairly small differences" or "they do not differ so much." The coded protocols were used to test theoretical predictions concerning whether a lexicographic choice rule or an addition of utility differences procedure was employed by the subjects. Finally, verbal protocols have been used by a number of researchers as a basis for computer models of consumer behavior (e.g. Bettman, 1970; Haines, 1974; Payne, 1976). A good discussion of the prospects and problems of such models may be found in Bettman (1974). A more extensive discussion of verbal protocols and other process tracing methods is provided in Payne, Braunstein, and Carroll (Note 3).

The next section of the paper will illustrate the use of verbal protocols, and other more traditional marketing tools (questionnaires, consumer purchase diaries), to study the behavior of consumers shopping in supermarkets.

SUPERMARKET SHOPPING BEHAVIOR

[The research reported in this section of the paper was conducted by Robert Blattberg, Harry Davis, and the two authors, at the University of Chicago.]

Method

The subjects were a convenience sample of 19 housewives with children in the five to twelve year old age bracket. The subjects all lived in Chicago area suburbs. The firm sponsoring the study, a candy manufacturer, set the condition that the housewives have school-age children; previous marketing research indicated that consumers meeting this requirement were most likely to be heavy purchasers of candy. Shoppers were recruited by offering a $30 payment to each individual who completed all the study requirements. Each shopper also received a small gift part way through the study.

Each shopper supplied verbal protocols during six "major" weekly shopping trips to the supermarket. The shoppers were accompanied by female graduate students acting as observers. At the beginning of each trip, the observers read a standard set of instructions to the subjects:

"We want you to think aloud while shopping. Say what you are doing and where you are going in the store. Whenever you see something as you are walking through the store, mention it. Talk about the products and brands that you are buying as well as those you notice but do not buy. For instance, say what observations you are making about brands, their location in the store, package size, price, ingredients, flavors, and so on. Remember, say anything that comes to your mind no matter how unimportant it seems to you."

When necessary, the observers also made use of a set of standardized prompts to remind the subjects to continuously think aloud:

1. "Remember, say anything that comes to your mind, no matter how unimportant it seems to you."

2. "Say what you are doing now."

3. "Remember talk about whatever you see."

4. "Talk about what observations you are making."

5. "Mention anything that you are thinking about."

6. "Be sure to say what you are doing now."

Great care was taken to minimize the amount of prompting and the amount of interaction between observer and subject.

Finally, in order to familiarize the subjects with the procedure of thinking aloud, they were asked to give verbal protocols while examining a small booklet containing full page magazine advertisements. This task was completed at the subject's home prior to the first shopping trip.

A great deal of information was acquired in addition to the verbal protocol data. Demographic and shopping behavior data were obtained from a small questionnaire completed by the subjects when they were recruited. Consumer purchase diaries were maintained during the six-week period of in-store observation and during a two-week period either immediately preceding or following the observation period. While in the store, the observers recorded the following information about the shopper's behavior: (1) the path taken through the store; (2) any items taken from major point-of-purchase displays (free standing or end-of aisle displays); (3) the use of a shopping list (all shopping lists were collected by the observers); (4) the use of coupons; and (5) the shopper's nonverbal behavior when passing the candy department.

The methods used in our study extend the previous efforts to study shopping behavior using protocols along several dimensions (e.g., Bettman, 1970; King, 1969; Haines, 1974). For example, Bettman (1970), in perhaps the best known protocol study in marketing, used protocols obtained from only two shoppers. The protocols were used to develop a model of each subject in the form of a decision net or tree. Our study utilized a much larger sample of shoppers. In addition, protocols were obtained over a large number of shopping trips. Also, unlike the Bettman study, the subjects in the present study were not known personally by the researchers, and interactions were kept to a minimum during the shopping trip, e.g., use of only a structured set of prompts. We also did not attempt to build an idiosyncratic model of each subject. Finally, our use of a variety of data collection methods enabled us to cross validate our results. We strongly believe that a multi-method approach is essential to the study of consumer behavior processes.

Results

The final four verbal protocols for each shopper were treated as the basic data base. The first two shopping trips were considered to be practice trials.

The first step in the protocol analysis was to make a complete transcript of the verbal reports given by each shopper. The next step was to break the protocols up into single statements or phrases. According to Newell and Simon (1972), each phrase should correspond to a naive assessment of what constitutes a single task assertion or reference by the subject. Breaking up protocols in this way serves to clarify the sequential nature of the observations and to create units of analysis. As an example of this procedure, excerpts from one protocol are presented below. Complete transcripts can be obtained from the second author.

D1: OK, alongside the window, we have Heritage House mayonnaise.

D2: and it is on sale this week.

D3: and it is only 79 cents;

D4: and I'm going to take one quart.

D5: Now I am going to buy tuna fish,

D6: that is Starkist tuna fish;

D7: and it is 49 cents a can;

D8: I am going to buy six cans;

D9: OK, 2, 4, 6 Starkist tuna fish, OK.

D10: Along the window they have the specials

D11: Welch grape juice

D12: Oh, they have the great big bottles.

D13: $1.09.

D14: I did not know that they had the big ones.

.

.

.

D152: Now, there is the apple juice

D153: They like that

D154: Hawaiian Punch

D155: it is $2.47

D156: usually they have the orange juice over here

D157: tea bags

D158: They have their own brand

D159: $1.39

D160: I bought National tea bags

D161: and they were pretty good

D162: I kind of like them

D163: so I am not about to switch

.

.

.

D429: Let us look at some of this zucchini squash,

D430: and then that is it.

D431: That is not too bad.

D432: Check over and see what we have on the list,

D433: and see if they have red cabbage.

D434: That is it on the list.

D435: OK, that is about it for today.

Once the protocol has been broken up into small phrases, a major issue in protocol analysis is whether the verbalizations should be further encoded into formal categories. Newell and Simon (1972) argue that encoding the verbalizations can result in a substantial loss of information since language is already a device for encoding information. Consequently, they emphasize dealing with a protocol in the form of small phrases. Although treating protocols at this level of analysis can prove valuable in consumer research (e.g., Payne, 1976), a detailed and formal analysis of verbal data will probably require some form of encoding the verbalizations.

In the present study, each phrase in a protocol was coded in terms of a product category code and a process code. [The results to be presented are based on a partial analysis of our data base-35 protocols reflecting the behavior of 10 shoppers. The protocol analyses are based on approximately 20,000 coded statements.] There were 41 separate product category codes, generated from lists of product categories found in the supermarket trade journal Progressive Grocer. Because not all statements referred to a particular product category, three additional codes were used to reflect shopper movement statements, general comments about supermarkets, and miscellaneous statements.

The statements were also assigned one of 20 process codes grouped under five general headings: (1) statements of goals, needs, or strategies, e.g., "Do I need some butter"; (2) statements of simple awareness, such as item identification or item location, e.g., "There's the 2% milk"' (3) statements concerning information processing within product categories (levels of product attributes, absolute evaluations of product attributes or products, comparison of products), e.g., "This is 3 for $.99." "This is too sweet." Coke tastes better than Pepsi."; (4) product choice decisions, e.g., "OK, I'll buy the M&M's with peanuts."; and (5) miscellaneous statements.

The coding scheme used was based on two considerations-First, any coding strategy must reflect the nature of the task environment. In our case, coding the statements in terms of product categories emphasizes the organizing principles of all supermarket environments. Second, a scheme for coding must also reflect the theoretical interests and concerns of the researchers.

For example, two important issues affecting the coding of statements were the amount of absolute evaluation versus comparative judgment and the amount of evaluation involving self and involving others. Absolute evaluations against internal or external standards and direct comparison of alternatives on their respective attributes has been identified as a crucial distinction in decision theory (Pitz, 1975; Payne and Braunstein, Note 4). The potential importance of viewing the shopper as a family purchasing agent has been stressed by Davis (1976). Consequently, our coding scheme distinguished between statement referring to the shopper's belief and those referring to the beliefs of others. Other aspects of the coding scheme, including the emphasis on shopper strategies, were derived from similar considerations- The amount of attention given to different product categories was measured by the number of times an attribute was mentioned with reference to the product category. Across all shoppers, the product categories with the largest number of references included: fresh meat, fresh fruits and vegetables, dairy products, paper products, canned fruits and vegetables, cookies and snacks. The categories with the fewest mentions included canned meats, pet foods and supplies, specialty foods, and candy, gum, and nuts. A ranking of product categories by the number of attribute references proved to be similar to the ranking of product categories by the percentage of shoppers purchasing in each category (see Progressive Grocer, January, 1964; October, 1975).

Table 1 presents a breakdown of the product attributes mentioned by the shoppers. Notice that the most frequently mentioned product attribute was price. This was true for all shoppers and for all but three product categories (specialty foods, breakfast cereals, and desserts). This result supports earlier studies that indicate price sensitivity among shoppers (e.g., Russo, Krieser, and Miyashita, 1975). However, the result conflicts with studies that have found shoppers to be poor estimators of product prices (see Progressive Grocer, 1974). One possible explanation is that shoppers do not store all price information in memory (where it can be retrieved when asked for prices in an interview before or after shopping). Instead, shoppers make use of the store environment (with prices displayed on every package) as form of external memory.

TABLE 1

PRODUCT ATTRIBUTES MENTIONED DURING GROCERY SHOPPING

This explanation is only speculative, but it does point up one of the possible advantages of in-store protocols as opposed to shopper interviews.

Table 2 presents a summary of the types of process statements. The results appeared to be reasonably stable across individual shoppers. Measured in terms of amount of verbal activity, shoppers made relatively little effort comparing brands within product categories while shopping (2.1% of all statements). One possible interpretation of this result is that while in the store, shoppers may decide whether or not to purchase in a particular product category (e.g., to buy rice for dinner), but once the decision is made, the brand choice is more or less routine. A related interpretation is that a substantial amount of shopper behavior in supermarkets may involve processing at the product choice level in a shopper's goal hierarchy, and only a limited amount of processing may occur at the brand choice level.

The protocols also provided evidence supporting the concept that housewives seem to act as purchasing agents for their families. For example, Table 3 present excerpts from the protocols relating candy buying to other family members.

Additional information on shopper behavior was obtained through direct observation and purchase diaries. Table 4, for instance, presents the relationships between the use of a shopping list and the number of items and amount purchased. The data presented in Table 4 are based on the entire sample of 19 shoppers. Notice that when lists were used, only approximately 41% of the items purchased were found on the list. The percent of items on the list to total items bought per trip ranged from 22% to 76% across shoppers.

TABLE 2

NATURE OF VERBAL ACTIVITY DURING SHOPPING (BY INDIVIDUAL SHOPPER)

TABLE 3

EXAMPLES OF SHOPPER COMMENTS RELATING CANDY BUYING TO OTHER FAMILY MEMBERS

TABLE 4

RELATIONSHIP BETWEEN USE OF SHOPPING LIST AND AMOUNT PURCHASED (BY INDIVIDUAL SHOPPER)

The extent of buying from end-of-aisle and free standing displays was also determined by combining data from direct observation and the record of purchases. It was found, for example, that taking all shoppers together, 14% of all purchases were from displays. Looking at individual shoppers, the proportion of items purchased from displays ranged from 1% to 25%. However, these results should be interpreted with caution. The results may reflect differences among shoppers in terms of strategies, or could be due to differences among the stores' use of point-of-purchase materials.

SUMMARY

The results presented here illustrate how verbal protocols, in conjunction with other, more traditional marketing research tools, provided information about consumer decision behavior at a level of detail difficult to achieve otherwise. Data was obtained on the goals and attributes used by supermarket shoppers, e.g., price awareness; the choice processes used by shoppers, e.g., amount of brand comparison; and the effect of in-store and social influences on shopping behavior. The collection of such data, along with other forms of high density, sequential data seems essential if we are going to develop and test complex and dynamic models of consumer behavior.

With regard to other process tracing techniques, there are differences of opinion as to whether such methods as eye movement recording or explicit information search procedures, should be preferred over verbal protocols as a method of collecting process data. Russo and Rosen (1975) argue that eye-movement records are better because "They are unobtrusive, detailed, and difficult to misrepresent." These claims may be true, although it is not clear how unobtrusive eye-movement recording has been given the apparatus used. Simon (1976), on the other hand, has argued that "verbal protocols provide a rich source of data at densities comparable to the densities of eye-movement records, and containing far more information per data than the latter" (p.261).

Jacoby (1976) has argued even more strongly for the use of information search techniques as opposed to verbal protocol procedures. He apparently feels that information search data are "behavioral" whereas verbal protocol data are not to be trusted. Jacoby does not seem to make any distinction between verbal protocols and other types of verbal data, such as responses to post-decision questionnaires and interviews. However, as was noted earlier, there are important distinctions between verbal protocol data and post-decision verbal reports. These distinctions make verbal protocols a much more valid source of process information.

One problem with the information acquisition methods is that they focus exclusively on the subject's use of objective, external information. The method does not easily allow for insights into an individual's use of information stored in internal memory. This problem relates to Simon's (1976) statement that verbal protocols provide more information about cognitive processes than eye-movement records.

Another problem is that information acquisition methods provide little or no indication of when and if the information being acquired is actually being processed. Payne and Braunstein (Note 4), for example, found one subject who (according to a monitoring of the subject's acquisition of information) appeared to be consistently searching all available information in an interdimensional fashion. The search appeared to be independent of all number of alternative gambles presented. Such a pattern of information use suggested that some form of an additive decision rule was being employed by the subject (see Payne, 1976). However, examination of the protocol showed that some of the acquired information was ignored by the subject and that the amount of information (number of attributes or dimensions) considered per alternative varied across alternatives; this pattern suggested a conjunctive type of model. Excerpts from the protocol made it clear that the subject had adopted the practice of obtaining all the available information before proceeding with the decision process. The ability of verbal protocols to detect such activity represents one of the greatest advantages of protocol over eye-movement and explicit information acquisition procedures.

Finally, information acquisition studies usually require the decision task to be more structured. This may present problems as consumer researchers attempt to do more "real-world" research.

Obviously, the preferred research strategy would be to employ more than one process tracing technique. Examples of such a multimethod approach to research can be found in Payne and Braunstein (Note 4) and Russo and Dosher (Note 2). Where a multimethod approach is impossible, the researcher will have to choose among process tracing methods on the basis of theoretical concerns, the availability of apparatus, and the relative difficulty in analyzing data collected by each method. The researcher faced with this choice can find encouragement in the results of efforts to compare methods. Newell and Simon (1972), for example, present evidence of a high degree of convergence in the interpretations about problem solving based on eye-movement data and on verbal protocols.

Finally, while we have argued the values of verbal protocols, it is clear that there are definite costs associated with the use of such a process tracing method. Protocols for example, provide researchers with a large amount of detailed data, which at present cannot be simply run through a computer program for analysis. While procedures for examining protocol data are currently being developed, it is clear that an investigator choosing to do a protocol study will probably have to settle for an intensive investigation of a relatively small number of subjects. However, as illustrated by the efforts to investigate contingent processing in decision making (Payne, 1976; Payne & Braunstein, Note 4), it does appear that process tracing techniques can lead to generalities about behavior, in spite of the limited number of subjects run in any particular experiment.

Another problem with using verbal protocols is that standard summary statistics for such data are not well developed. Consequently, a researcher using such methods is often forced to present the results of his study in great detail, e.g., parts of the protocols themselves. Haines (1974), in fact, advocates such a procedure. Finally, it is obvious that verbal protocols represent an obtrusive measure of behavior. We have examined the obtrusiveness of the verbal protocol procedure as part of two studies of decision making (Carroll & Payne, 1977; Payne & Braunstein, Note 4). Our results seem to indicate that the verbal protocol procedure may slow down the decision process slightly but does not change it fundamentally. A similar effect for verbal protocols has been suggested by other researchers (e.g., Dansereau and Gregg, 1966; Newell and Simon, 1972). However, much more research is needed to examine directly the effects of various process tracing data collection methods on the behavior exhibited by subjects.

In summary then, verbal protocols are a valuable source of data for consumer research. However, it does not appear that by themselves they will "Save The World."

REFERENCE NOTES

1. Svenson, O. Coded think-aloud protocols obtained when making a choice to purchase one of seven hypothetically offered houses: Some Examples. Progress report on project of cognitive processing in decision making. University of Stockholm, 1974.

2. Russo, J. E., Dosher, B. A., Dimensional evaluation: A heuristic for binary choice. Unpublished manuscript, University of California, San Diego, 1975.

3. Payne, J. W., Braunstein, M. L., & Carroll, J. S., Exploring predecisional behavior: An alternative approach to decision research. Unpublished manuscript Graduate School of Business, University of Chicago, 1977.

4. Payne, J. W., & Braunstein, M. L. Contingent processing in risky choice: A process tracing investigation. Unpublished manuscript, Graduate School of Business, University of Chicago, 1977.

REFERENCES

Bettman, J. R. Information processing models of consumer behavior. Journal of Marketing Research, 1970, 7, 370-376.

Bettman, J. R. Decision-net models of buyer information processing and choice: Findings, problems, prospects. In G. D. Hughes and M.D. Ray (Eds.), Buyer/consumer information processing. Chapel Hill, North Carolina: University of North Carolina Press, 1974.

Carroll, J. S., & Payne, J. W. Judgments about crime and the criminal: A model and a method for investigating parole decisions. In B. D. Sales (Ed.), Perspectives in law and psychology. (Vol. 1): The criminal justice system. New York: Plenum, 1977.

Dansereau, D.F., & Gregg, L. W. An information processing analysis of mental multiplication. Psychonomic Science, 1966, 6, 71-72.

Davis, H. D. Decision making within the household. Journal of Consumer Research, 1976, 2, 241-260.

Haines, G. H. Process models of consumer decision making. In G. D. Hughes & M. L. Ray (Eds.) Buyer/consumer information processing. Chapel Hill, North Carolina: University of North Carolina Press, 1974.

Jacoby, J. The emerging behavioral process technology in consumer decision making research. In W. D. Perreault, Jr. (Ed.), Advances in consumer research (Vol. 4). 1977, 263-265.

King, R. H. A study of two problems of building a model to simulate the cognitive processes of a shopper in a supermarket. In G. H. Haines (Ed.), Consumer behavior: Learning models of purchasing. New York: The Free Press, 1969.

Montgomery, H. A. A study of intransitive preferences using a think aloud procedure. In H. Jungerman, & G. de Zeeuw (Eds.), Proceedings of The Fifth Research Conference on Subjective Probability, Utility and Decision Making, 1976.

Newell, A., & Simon, H. A. Human problem solving. Englewood Cliffs, New Jersey: Prentice-Hall, 1972.

Nisbett, R. E., & Wilson, T. D. Telling more than we can know: Verbal reports on mental processes. Psychological Review, 1977, 84, 231-259.

Payne, J. W. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 1976, 16, 366-387.

Pitz, G.F. Decision making and cognition. In H. Jungerman & G. de Zeeuw (Eds.), Proceedings of The Fifth Conference on Subjective Probability, Utility, and Decision Making, 1976.

Russo, J. E. Krieser, G. & Miyashita, S. An effective display of unit price information. Journal of Marketing, 1975, 39, 11-19.

Russo, J. E., & Rosen, L. D. An eye fixation analysis of multialternative choice. Memory & Cognition, 1975, 3, 267-276.

Simon, H. A. Discussion: Cognition and social behavior. In J. S. Carroll & J. W. Payne (Eds.) Cognition and Social Behavior, Hillsdale, New Jersey: Erlbaum, 1976.

Tversky, A. Intransitivity of preferences. Psychological Review, 1969, 76, 31-48.

----------------------------------------