Keyword Recognition: a New Methodology For the Study of Information Seeking Behavior

ABSTRACT - The traditional field and laboratory methodologies for the study of external information search behavior are reviewed. A new computer methodology, which reduces the structure placed on the search task, is introduced and its advantages and disadvantages are discussed. The results from a preliminary study suggest that this methodology is both a useful and feasible approach for the study of information seeking behavior. In addition, this computer methodology overcomes some of the problems associated with methodologies used in the past.

Citation:

Julie L. Ozanne (1988) ,"Keyword Recognition: a New Methodology For the Study of Information Seeking Behavior", in NA - Advances in Consumer Research Volume 15, eds. Micheal J. Houston, Provo, UT : Association for Consumer Research, Pages: 574-579.

Advances in Consumer Research Volume 15, 1988      Pages 574-579

KEYWORD RECOGNITION: A NEW METHODOLOGY FOR THE STUDY OF INFORMATION SEEKING BEHAVIOR

Julie L. Ozanne, Virginia Polytechnic Institute and State University

ABSTRACT -

The traditional field and laboratory methodologies for the study of external information search behavior are reviewed. A new computer methodology, which reduces the structure placed on the search task, is introduced and its advantages and disadvantages are discussed. The results from a preliminary study suggest that this methodology is both a useful and feasible approach for the study of information seeking behavior. In addition, this computer methodology overcomes some of the problems associated with methodologies used in the past.

INTRODUCTION

Because of search behavior's importance in consumer behavior, public policy, and managerial decision making, it is one of the most important and heavily researched areas of choice behavior. Field studies dominated the early research in this area; however, many problems exist with survey methods used in these studies. Later research in the laboratory relied heavily on the information display board methodology, which overcomes many of the problems associated with field studies. Nevertheless, this method also has drawbacks. This study proposes a new computer methodology, a keyword recognition problem, that minimizes many of the problems associated with both the survey and the information display board methods, while offering significant advantages.

PAST METHODOLOGIES

Field Studies. One of the most frequently replicated findings in field research is the apparent lack of information search by consumers, even when the purchase involves a large outlay of money (Dommermuth 1965; (Katona and Mueller 1955; Newman 1977; Newman and Staelin 1972; Udell 1966). These studies use many measures to capture information search behavior. (1) the number of stores visited, (2) the number of shopping trips before a purchase, (3) the amount of time taken in the store, (4) the number of information sources used, (5) the number of types of information sources used, and others (Katona and Mueller 1955; Newman 1977; Newman and Lockeman 1975; Newman and Staelin 1972). Most of these measures are used as surrogates for search behavior and, consequently, do not necessarily reflect the actual number of information bits received by the consumer. For example, the amount of information search represented by one visit to a store is unclear (Newman 1977). Also, field studies treat the sources of information equivalently, yet some of the sources must provide more information than others.

Newman and Lockeman (1975) cite additional limitations of the survey method: (1) the inability to capture the details of the search, (2) the consumers' limited responses to survey questions, and (3) the incomplete recall by the subjects. In an attempt to overcome these limitations, Newman and Lockeman employed a combination of survey-based and observational measures. They found that a low correlation existed between the survey-based and observational measures. Moreover, considerable search behavior occurred in the retail store, indicating that without the in-store observational measures, one would incorrectly conclude that a consumer's search for information is limited.

Most of the field studies rely on consumers' self-reports about their pre-purchase information search. These self-reports were given anywhere from a few days to two years after the purchase. Yet in the Newman and Lockeman study, subjects could not accurately recall their in-store information search behavior, even a few minutes later. As well, subjects may favorably bias a self-report if they believe they are being evaluated.- Given these limitations, the results from field studies in this area should be interpreted cautiously.

Laboratory Studies As opposed to field studies, laboratory studies offer an opportunity to study information search behavior in a controlled setting. In a laboratory setting one is not forced to rely on self-reported data; that is, the actual information seeking behavior can be measured directly. Furthermore, more detailed data can be generated.

Most of the studies in this area have relied on the same behavioral process methodology, presenting information--usually organized in an attribute-by-brand matrix--through the use of an information display board (Jacoby et al. 1976). The subjects acquire information by exploring the brand/attribute cells that interest them. This methodology is easy to implement and is a substantial improvement over the post-purchase self-reports used in the field studies; that is, the board methodology allows information search behavior to be measured directly. The information display board (IDB) methodology has led to a fruitful stream of research and variations in the basic methodology have appeared (Payne 1976a, 1976b). For instance, Hoyer and Jacoby (1981) computerized the task and expanded the 2-dimensional matrix into a 3-dimensional matrix by adding a source of information dimension.

Unfortunately, the IDB methodology, in all its forms, has some drawbacks. For example, Brucks (1984) argues that the board imposes a structure on a problem that in reality is often poorly defined; the buyer may not know how many alternatives are available or which attributes are important. Second, limited costs of search exist for the subject with the information display board. Third, in the real world such information is usually structured by brand. The board facilitates both processing by brand and processing by attribute. In addition, Arch, Bettman and Kakkar (1978) found that the task characteristics of a study can influence the search behavior of the subjects; that is, the information presentation format influences the information acquisition strategy. Given these limitations, it is not clear how closely search behavior with the information display board would reflect search behavior in a natural setting.

Brucks (1984, 1985) provides an alternative laboratory methodology-a computerized shopping simulation-that avoids many of the problems of traditional approaches. Like the IDB approach, this method allows actual search to be measured. However, unlike the IDB methodology, subjects can generate their own questions and attributes without having a structure imposed on their choice domain. The subjects access a database by typing questions into a computer terminal, while a video camera is filming their computer screen. -A hidden human interpreter watches a computer monitor and responds to the subjects' questions by typing in an identification code associated with the correct preprogrammed response. This method has advantages over the traditional methods; specifically, the unstructured choice has greater ecological validity than the IDB methodology. However, from a practical standpoint, this approach is difficult to employ because subjects must be run individually.

KEYWORD RECOGNITION APPROACH: A NEW COMPUTER METHODOLOGY

The keyword recognition program, as with the video monitoring approach, also avoids an unrealistic structuring of the choice domain. However, this approach is an artificial intelligence technique for interfacing with a user. Communication between a human and a computer using this technique is based on a few simple ideas. Subjects directly type into the computer their requests for information. The text is first scanned for keywords. When keywords are identified, the sentence is transformed according to a rule that is associated with the keyword and, based on the rule, the computer replies back to the subject. This is the concept behind computer programs that use natural language; however, in practice the procedure is much more complex (see Weizenbaum 1966). The most insurmountable obstacle with a natural language program is generating all the possible rules for multiple-key word sentences.

AD example might help to illuminate the problems involved with natural language programs. Say, for instance, that a subject wanted to find the gas mileage for Car A. The first difficulty would be generating all the possible ways that this question could be asked: 1) What is the gas mileage?, 2) What is the gas mileage per tank full?, 3) How long can this car go on a tank of gas?, and so on. The next problem would be creating rules for all of the different combinations of keywords. Rules would have to be generated for each basic sentence structure and for all of the different combinations of keywords.

Nevertheless, this keyword recognition approach can be made feasible by limiting the number of sentence structures that are used. If the subject is limited to using, for example, only one out of five sentence structures ("What type of ___ does Car A have?") and only one to three keywords used simultaneously, then the keyword program becomes feasible. With this constraint placed on the question format, it is also possible to use the keyword recognition approach on a micro computer. It is still necessary, however, to go through a lengthy process of generating all of the possible keywords and synonyms. Furthermore, one should also consider likely misspellings (for example, 'gas milage'). But this approach has several advantages over the IDB approach. The subject is still free to generate any questions that he/she wants to ask. The keyword recognition program does not structure what the subject asks, only how the questions are asked. It does not force subjects to search by either brand or attribute; that is, the task structure does not dictate the search strategy.

With any computer methodology, it is important to include warm-up exercises for the subject. On the one hand, a subject may fear working on the computer. The warm-up session should alleviate these fears. On the other hand, a subject may be intrigued by the novelty of the computer. In this case, the warm-up session should allow some of this novelty to "wear-off". An example of a warm-up exercise is presented in the next section.

Both Brucks' methodology and the new methodology offer several advantages over the traditional process methodologies. For example, it was found in debriefing subjects that they were naive with respect to the computer's capabilities; that is, the computer allows for an unobtrusive trace of the unsuspecting subjects' search processes. In addition, since the subjects do not believe their actions are being traced, they are less likely to monitor themselves and alter their behavior. Similarly, the use of the computer should diminish the subject/experimenter interaction and, thus minimize the potential threats to construct validity--specifically, evaluation apprehension and experimenter expectancies (Cook and Campbell 1979). By controlling experimental conditions that normally would vary (the facial expressions and the tone of the questioner, for example), the computer also aids in promoting a consistent experimental setting.

Many costs of information search exist: time, money, frustration and others. With these computer methodologies it is easy to create a cost of information search. Time delays can be programmed into the computer. Although time is not the only cost of search, it does represent an actual cost of search.

The computer methodologies do have problems. There is no way to control or measure internal search. (This criticism can also be made of the other methodologies.) As well, the extent to which behavior in the laboratory reflects behavior in a natural setting is not clear.

Theoretically, both computer methodologies are superior to the information display board methodology because they reduce the structure placed on the task. From a practical standpoint, the major advantage of the keyword recognition program is that the data collection is easier, faster, and cheaper. With the high availability of personal computers, subjects can be run concurrently, thus increasing statistical power.

USING THE KEYWORD RECOGNITION PROGRAM: A PRELIMINARY STUDY

The keyword recognition program was tested empirically in a study of information seeking behavior for automobiles (author). This was a fairly strict test of the methodology. An automobile is a complex product with hundreds of attributes. (About 200 attributes and over 750 synonyms were stored in the keyword data base). If the methodology can be used for an automobile, then it should also be viable for the simpler products with fewer attributes.

Description Upon entering the study, subjects were seated before a computer terminal. All of the directions and the warmup exercises were already programmed into the computer. The subject merely had to strike any key to read the next screen of instructions. The subjects first engaged in a warm-up exercise until they were comfortable using the computer. See Exhibit 1 for the actual warm-up exercise.

When the subjects understood how to use the computer, they advanced to the actual study. The computer procedure used in the warm-up exercise parallelled the procedure in the automotive study. (See Exhibit 2 for the directions and an example of an actual computer screen in the automotive task.)

For example, if the subject wanted to know the gas mileage for car A the subject had to enter a "3" (then 'What type (kind) of ___' appeared on the computer screen), type in "M.P.G." or a synonym, hit enter (then 'of car ___' appeared on the screen), and, finally, type in the letter"A". If the subject used a keyword that was not recognized by the program then the subject received a message instructing him/her to ask the monitor for a synonym that the computer would recognize.

An Evaluation of the Methodology. Several criteria were used to determine the viability of this methodology. First, the subjects must understand the task and feel comfortable using the computer. A four-item scale measured task understanding (Cronbach's coefficient alpha=.72). Prom an examination of the scale mean, it appears that subjects did understand the task. On a 7 point scale, '1' meaning the subject understood the task, the mean was 1.93 with a standard deviation of .91 (see Exhibit 2).

In addition to understanding the task, subjects needed to be involved in the search. Automobiles are a high involvement product so it was important from a theoretical standpoint for subjects to be involved in the task. Task involvement was measured using a five-item scale (see Exhibit 2). The coefficient alpha for this scale was .76. From an examination of the mean, subjects exhibited a high level of involvement (mean = 1.88 and standard deviation = .81).

The third criterion used to evaluate the methodology was an analysis of the number and type of errors made by the subjects. The first measure of errors was the average number of errors; that is, any request made by a subject that was initially unanswered fell into this category. This measure was the total number of unanswered questions divided by the total number of requests made.

The average number of errors was broken down into three subcategories: typographical errors, "okay" errors, and "bad" errors. Typographical errors included such errors as mistyped words, misspelled words, accidental carriage returns,.. These requests were considered normal, unavoidable errors, which do not reflect poorly on the methodology. The "okay" errors were requests that were made, went unanswered, but were subsequently answered. Either the subjects asked the monitor for a synonym or the subjects thought of a synonym by themselves. Again, these errors were not detrimental to the methodology because the subject was able to get the desired information. The "bad" errors were any requests for information that went unanswered. Although these "bad" errors are not detrimental to the methodology per se, they did reflect an inadequacy in the data base; and, if the data base is incomplete, then the methodology is limited. As well, the subjects could become frustrated by their inability to ask questions and this frustration could alter the results of the study.

There were thirty-three different words that were not recognized by the computer program yet were valid requests. Many of these words were synonyms for the attributes already stored in the data base; for instance, "transmission" was stored in the data base with several synonyms, however, the synonyms "gears", "gear shift" and "five gears" were not stored. Some of the words should have been in the data base but were overlooked: "gas tank," "tank," and "muffler." Finally, the majority of "bad" errors were unusual combinations of words, which may have been difficult to generate a priori: "color scheme," "color interior" and "maintenance needs."

The measure of the number of bad errors is conservative measure. For example, a subject could have used five synonyms to ask about the same attribute and this would have been counted as five bad errors, when the subject had only failed to discover the value of a single attribute. Nevertheless, five unanswered questions could certainly lead to frustration.

The number of requests that were mistakes was 15.4% of the total number of requests. Of this total, 8.0% were typographical errors, 3.4% were "okay" errors and 4.0% were "bad" errors. Overall, this rate seemed to be an acceptable level of nonresponse errors given that this was the first use of the methodology and a complex product was used. Of course, without a baseline number of errors, absolute acceptability cannot really be determined. It should be noted, however, that in subsequent studies the error rate would decrease if the unrecognized words were added to the data base. As well, if this methodology was used with less complex products then the error rate should decrease substantially. Clearly initial results are good enough to warrant additional exploration of this methodology. However, additional empirical research is needed to explore the limitations of his methodology.

CONCLUSIONS

Some of the field and laboratory methodologies for the study of information acquisition were reviewed. A new computer methodology was introduced, which overcomes many of the problems associated with the past methods. This paper argues that the keyword-recognition approach, like the computerized shopping simulation approach, reduces the unrealistic structuring of the choice domain. Furthermore, a preliminary study indicates that the new approach is feasible.

Future studies might consider using the IDB and computer methodologies to study the same phenomenon. It would be useful to compare results from these methods. For example, because the computer methodologies do not impose a structure on the search task, one would suspect that data generated from the computer methodologies on the subjects' search strategy would correlate more with actual behavior. However, a fundamental question that all laboratory research must face is do the results generated in an artificial, controlled environment match the results generated in a natural setting. Clearly some of the artificial dimensions of the lab context, such as being unable to actually see and use the product, must impact the behavior exhibited by the subjects. As well, only one source of information is employed, yet realistically subjects rely on a variety of information sources.

EXHIBIT 1

COMPUTER WARM-UP EXERCISE

EXHIBIT 2

AUTOMOTIVE TASK

EXHIBIT 3

SCALES USED TO CHECK METHODOLOGY

REFERENCES

Arch; David C, James R. Bettman, and Pradeep Kakkar (1978), "Subjects' Information Processing in Information Display Board Methodologies," in H. Keith Hunt, (ed.), Advances in Consumer Research. Volume V, Ann Arbor Association for Consumer Research, 555-560.

Brucks, Merrie L. (1984), "An Experimental Methodology to Assess the Influence of Prior Knowledge on Information Search," in Developments in Marketing Science. Volume VIII, Jay D. Lindquist, ed, Kalamazoo, Michigan: Academy of Marketing Science, 87-89.

Brucks, Merrie L. (1985), "The Effects of Product Class Knowledge on Information Search Behavior," The Journal of Consumer Research, 12, 1-16.

Dommermuth, William P. (1965), "The Shopping Matrix and Marketing Strategy," Journal of Marketing Research, 2, 128-132.

Cook, Thomas K. and Donald T. (1979), Quasi-Experimental Designs for Research, Chicago: Rand McNally.

Hoyer, Wayne D. and Jacob Jacoby (1981), "Three Dimensional Information Acquisition: An Application to Contraceptive Decision Making," in Advances in Consumer Research, Volume 8, ed. Kent B. Monroe, Ann Arbor, Michigan: Association of Consumer Research, 618-623.

Jacoby, Jacob, Robert W. Chestnut, Karl C. Weigl, and William Fisher (1976), "Pre-Purchase Information Acquisition: Description of a Process Methodology, Research Paradigm, and Pilot Investigation," in Beverlee B. Anderson, (ed.) Advances in Consumer Research. Volume, 8, Ann Arbor Association for Consumer Research, 306-314.

Katona, George, and Eva Mueller (1955), "A Study of Purchase Decisions in Consumer Behavior," in Lincoln H. Clark, ed, Consumer Behavior: The Dynamics of Consumer Reaction, Volume 1, New York: New York University Press, 30-87.

Newman, Joseph W. (1977), "Consumer External Search: Amount and Determinants," in Arch G. Woodside, Jagdish N. Sheth, and Peter Bennett, eds, Consumer and Industrial Buying Behavior, New York: North Holland, Inc, 79-94.

Newman, Joseph W, and Bradley D. Lockeman (1975), "Measuring Prepurchase Information Seeking," Journal of Consumer Research, 2, 216-222.

Newman, Joseph W, and Richard Staelin (1972), Prepurchase Information Seeking for New Cars and Major Household Appliances," Journal of Marketing Research, 9, 249-257.

Payne, John W. (1976a), "Heuristic Search Processes in Decision Making, in Beverlee B. Anderson, (ed.), Advances in Consumer Research, Volume 8 Chicago: Association for Consumer Research, 321-327.

Payne, John W. (1976b), "Task Complexity and Contingent Processing in Decision Making: An Information Search and Protocol Analysis," Organizational Behavior and Human Performance, 16, (August), 366-387.

Udell, Jon G. (1966), "Prepurchase Behavior of Buyers of - Small Electrical Appliances," Journal of Marketing, 30, 50-52.

Weizenbaum, J. (1966), "ELIZA--A Computer Program for the Study of Natural Language Communication Between Man and Machine," Communications of the Association for Computing Machinery, 9, 36-45.

----------------------------------------