Computer-Controlled Experimentation in Consumer Decision Making and Judgment

ABSTRACT - Relative to other laboratory techniques for experimentation in consumer decision making and judgment, computers offer three major advantages to researchers: 1) potential for complex and dynamic task structures, 2) precise measurement and control of time, and 3) automation of experimental procedures and data file creation. Disadvantages are related to the artificiality of the task as well as user interface, software, and hardware issues. These advantages and disadvantages are discussed and directions for future research that is especially well-suited for computer-controlled experimentation are suggested.

Citation:

Merrie Brucks (1990) ,"Computer-Controlled Experimentation in Consumer Decision Making and Judgment", in NA - Advances in Consumer Research Volume 17, eds. Marvin E. Goldberg, Gerald Gorn, and Richard W. Pollay, Provo, UT : Association for Consumer Research, Pages: 905-909.

Advances in Consumer Research Volume 17, 1990      Pages 905-909

COMPUTER-CONTROLLED EXPERIMENTATION IN CONSUMER DECISION MAKING AND JUDGMENT

Merrie Brucks, University of North Carolina

ABSTRACT -

Relative to other laboratory techniques for experimentation in consumer decision making and judgment, computers offer three major advantages to researchers: 1) potential for complex and dynamic task structures, 2) precise measurement and control of time, and 3) automation of experimental procedures and data file creation. Disadvantages are related to the artificiality of the task as well as user interface, software, and hardware issues. These advantages and disadvantages are discussed and directions for future research that is especially well-suited for computer-controlled experimentation are suggested.

INTRODUCTION

The growing availability and capability of the microcomputer has resulted in increased interest in computer-controlled experimentation in consumer decision making and judgment research (e.g., Brucks 1985; 1988; Brucks and Chakravarti 1986; Brucks and Schurr 1990; Drumwright 1986; Hoyer and Jacoby 1983; Johnson et al. 1986; Olshavsky and Rosen 1985; Ozanne 1988; Ozanne, Brucks and Grewal 1989; Painton and Gentry 1985; Rosen and Olshavsky 1987; Urbany 1986; Urbany, Bearden and Weilbaker 1988; Zeithaml and Brucks 1987). This article identifies major advantages and disadvantages of computer-controlled experiments in consumer decision/judgment research, and points out directions for future research that is especially well-suited for computer-controlled experiments.

ADVANTAGES OF COMPUTER-CONTROLLED EXPERIMENTS

Relative to other laboratory techniques for experimentation in consumer decision making and judgment, computers offer three major advantages to researchers: 1) potential for complex and dynamic task structures; 2) precise measurement and control of time; and 3) automation of experimental procedures and data file creation. Each of these advantages is discussed below.

Potential for Complex and Dynamic Task Structures

Typically, laboratory studies of information search behavior have examined only two factors: brands and attributes. Yet the information environment that consumers usually face is significantly more challenging. Consumers may obtain information of various types (e.g., attribute values, product class information, overall evaluations of alternatives) from various sources (e.g., brochures, advertising, salespersons, friends, consumer and hobbyist magazines) by various methods (phone, visit, incidental exposure). It is important to study decision making behaviors of consumers in complex information environments since it has been shown that information search and processing is contingent on task complexity (Payne 1976; 1982).

Menu-driven computer programs offer a method by which to study large and complex information environments. The sequence and contents of menus may be structured in such a way as to simulate many of the complexities of real-life decision making. For example, the subject may first be asked to choose an information source and method of contacting that source (phone a friend, visit a store, etc.) Next, the subject may choose what type of information to request (recommendation, specific information about a brand, advice on important attributes). After that piece of information has been received, the subject may choose to continue requesting information from that source or to go to another source.

Obviously, the more complex a simulation is, the more programming and computer memory will be required. One must carefully consider how much task complexity is necessary for a specific experiment's goals. While one might be tempted to incorporate as much complexity as possible in order to achieve "realism," this may not be desirable. It is impossible to simulate the real world exactly, so descriptive data resulting from such studies are always suspect. For example, if one wants to learn the percentage of people who use each of several information sources, the simulation would have to match the real world exactly on the perceived relative costs and benefits of contacting each of the information sources in order to produce meaningful results. If we understood search costs and benefits well enough to do this, we would not need the simulation to learn about information source usage.

The task complexity associated with computer simulations is most useful for experiments and quasi-experiments in which the effects of the independent variables are contingent on task complexity. For example, research in cognitive psychology has shown that one of the major effects of expertise is on problem perception and representation (e.g., Chase and Simon 1973; Chi, Glaser, and Rees 1981). Thus, some of the information processing advantage that experts have over novices may not be evident in simple, well-structured tasks. A complex, ill-structured information environment provides a better test of expertise effects (Brucks 1985).

In addition to large task structures with a variety of stimuli, computers are ideally suited for dynamic task structures, i.e., tasks where the stimuli or task objectives presented to the subject depend on the subject's previous behavior or current status in the task. A dynamic task structure is useful when the researcher wants to place a manipulation, measure, or instruction at a specific intermediate point in the task. For example, the effects of types and timing of unintentional exposure to information can be studied by interjecting various screen displays at predetermined points in the decision process. More specifically, the influence of personal selling, stockouts, and unintentional exposure to advertising on the consumer decision or judgment process may be more directly examined.

Measurement and Control of Time

Time may be measured to create dependent variables, or it may be controlled to create independent variables. As a dependent variable, time measurements are often used to infer amount of cognitive processing. For example, the amount of time it takes for an individual to agree or disagree with the statement "Car X is highly reliable" indicates whether this evaluation has been previously stored in memory (and must simply be retrieved) or whether an evaluation must be formed at the time the question is presented. Thus, response time measurements may be used to study memory content, memory structure, and also inference processes. In information search experiments, the time spent examining a piece of information, or information of a specific type (for example, the attribute price), indicates the amount of processing devoted to that piece of information or information type. Traditionally, response times have been measured with tachistoscopes - specialized equipment not always available to consumer researchers. The capability of microcomputers to measure response times makes this methodology widely accessible.

As an independent variable, time may be manipulated by varying the amount of time a stimulus or piece of information is visible (exposure time), by varying the amount of time before subjects' information requests are displayed on the screen (waiting time), and by varying the time allotted to the task (time pressure). Computers may be programmed to manipulate these time variables precisely and reliably within or between subjects.

In decision making experiments, waiting time may be used to represent search costs. The major advantage is that waiting time parallels the cost of search in actual purchase situations. In the real world of consumer behavior, information is collected by such activities as talking to friends, visiting showrooms, or making telephone calls. For most people the major cost associated with such activities is the time it takes to do them. Although most purchases are not made under explicit time pressure, consumers do have alternative ways to spend their time. Waiting time, then, represents the time cost of information search. A disadvantage of using waiting time to represent search costs, however, is that people have widely differing perceptions of the value of their time, and it is very difficult to control for this statistically or experimentally. Another disadvantage is that waiting in front of a blank computer screen may be much more or much less aggravating than spending time shopping in the real world. Other methods for operationalizing time costs may be used in conjunction with waiting time, such as providing subjects with opportunity costs for time spent doing the decision-making task.

Although microcomputers control time precisely and reliably, mainframe computers do not The number of other users and the jobs they are running affect the response time of the computer to the subjects' keyboard entries. Thus, time-sharing computer systems should not be used if time control is critical.

Automation of Procedures and Data Creation

Computers do exactly what they are programmed to do (assuming reliable hardware). Thus, multiple administrations of treatments and tasks will be identical each time, eliminating unwanted variations due to differences in experimental procedures. Furthermore, depending on hardware availability, many subjects may be run simultaneously - a big advantage for process-oriented experiments.

The data generated from process-oriented experiments are-notoriously difficult to analyze. Computers can eliminate the need to manually convert the subjects' process traces into a quantitative data file. For example, Search Monitor (Brucks 1988) contains a program that keeps track of specific types of subjects' actions and outputs these data directly into a file that is in the usual format for input to a commercial statistical package. With some minimal knowledge of computer programming, the program can be altered to keep track of additional output variables.

DISADVANTAGES

It is often argued that the primary disadvantage of computer-controlled experiments is their artificiality. Specifically, computer simulations are not realistic depictions of real-world decision making, and they might encourage a more rational approach to decision making and judgment than consumers usually use. As mentioned earlier, this artificiality is certainly an important obstacle to using computer-controlled simulation for descriptive research. It may or may not be an issue for experimentation, however. One must suspect that the artificiality of the task interacts with one of the independent variables in its effect on the dependent variables in order to argue that artificiality is a serious threat to external validity (see Lynch 1982 for an expanded discussion of this issue).

The remaining disadvantages are related to user interface, software, and hardware issues. User interface refers to the communication between the experimental subject and the computer. Difficulty may arise if subjects are unfamiliar with computers, or worse yet, intimidated by them. These problems are minimized if the task includes early screens designed to reduce subject anxiety and explain simple commands. Subjects need to know what will happen if they press the "wrong" key and they need a chance to practice skills that will be needed in the experimental task, such as choosing options from a menu. Difficulties may also arise if the task requires subjects to formulate and type in their own requests and commands (rather than choose them from a list). If the sample is drawn from a non-student population across various social strata, the resulting data set may contain excessive undesired variation due to differences in verbal and typing abilities of the subjects.

The expertise and time required to develop original software is a major disadvantage. Search Monitor (Brucks 1988) and Mouselab (Johnson et al. 1986) were developed by consumer researchers to handle task structures typical of consumer research experiments, thus avoiding the need to develop new software for every new experiment. Commercial user-interface software, such as HyperCard, also may be used instead of developing software from scratch. However, even these software packages take time to learn to use and to create specific experiments.

Hardware availability may also pose a problem. If a large number of subjects are required, it is helpful to have several computers (or terminals) available. Although it is certainly possible to run a one-computer study, it adds to the already lengthy process of implementing a computer-controlled study. Another potential hardware-related problem is the size and quality of the screen display. Computer screens are relatively small and may not be able to display simultaneously all the desired information or stimuli.

SUGGESTIONS FOR FUTURE RESEARCH

Almost any consumer decision-making task may be implemented on a computer, but it may not be appropriate to do so. For many research problems, the only major advantage of the computer is automation, which may not be important enough to offset the disadvantages - especially the time and expense of developing software for the task. In this section, some research problems that are particularly likely to benefit from a computer-controlled approach are discussed.

Complex Purchase Environments

The computer's capability to store and retrieve great quantities of information make it ideal for studying consumer information acquisition in complex purchase environments. It is relatively easy to include multiple stores, multiple information sources, and multiple purchase goals, as well as the usual assortment of brands and attributes in a decision-making task. Thus, for example, one can experimentally examine the relationship between store choice and brand choice or the effect of stores' inventory and pricing strategies on information search. Furthermore, the number of available stores, information sources, and purchase goals may contribute to perceived task complexity or information overload. Future research is needed to examine this possibility and its implications.

Ill-Structured Choice Tasks

Although much previous research has presented subjects with well-structured decision tasks, the consumer choice situation is often an ill-structured problem (Brucks and Mitchell 1981). Consumers do not usually know all the available alternatives and product attributes before beginning a decision-making task, nor are they certain which methods for obtaining information will be fruitful. It is suggested that consumers develop an internal representation of the decision problem based on these task characteristics during the decision-making task. What factors affect this process of problem representation? How do the (probably) concurrent processes of problem representation and problem solution interact?

Computer-controlled tasks may provide a method to examine these questions. By utilizing dynamic task structures, the task-defining characteristics do not have to be identified at the beginning of the decision process. To examine how problem representations are formed during decision making, the subject might be allowed to search for information on task-defining characteristics (e.g., number of alternatives, identity and importance of product attributes, number of available stores, and store characteristics, as well as attribute values. The amount and timing of search on task-defining characteristics might illuminate the development of problem representation, especially if coupled with concurrent verbal protocols.

Interrupts in Decision Making

The concept of interrupts is crucial to Bettman's information processing model of consumer choice (Bettman 1979). Bettman uses Simon's (1967) proposal that a scanner mechanism continually monitors the environment, looking for conditions that might require changes in current activities. When such conditions are found, the interrupt mechanism stops progress on current activities and initiates a response to the condition encountered. Bettman's model postulates that interrupts affect all parts of the decision-making process: motivation, attention, information acquisition and evaluation, decision processes, and consumption and learning processes. Yet little research has examined responses to interrupts and their effects on the consumer decision process.

Computer-controlled experiments can use dynamic task structures, which provide an ideal methodology for studying interrupts. Phenomena suspected of causing interrupts may be programmed into the task, presenting themselves to consumers at specific points in the decision process. Thus, consumers' responses to the timing and nature of various interrupts may be examined.

The major source of interrupts is a perceived departure of environmental conditions from those expected or anticipated. More specifically, interrupts result from impediments to goals, such as stockouts and unexpected negative information (e.g., high price). Interrupts may also result from unintentional exposure to information. For example, unsolicited advice from a salesperson or another consumer may cause a consumer to rethink a purchase intention. Similarly, unintentional exposure to advertising or a competitor's display may also cause an interrupt.

Salesperson Influences

Although information from salespeople is a major influence on consumer purchase decisions (Wilkie and Dickson 1985), it has received little attention in the information processing literature. Little is known about the effects of information obtained from salespeople on the process of decision making and judgment. Such influence may occur in the form of an interrupt (unsolicited information), or sales influence may be actively sought by consumers. One direction for future research is to examine the effect of various aspects of salesperson-customer interactions on the formation of problem representation as well as problem solution strategies.

The computer provides a way to experimentally manipulate the timing and content of sales messages while providing access to the consumer's decision process. One might argue that presenting information on a computer screen and ascribing that information to a salesperson does not sufficiently capture the complexities of the face-to-face interaction between customer and salesperson. While this is a real limitation, it does not imply that computer simulation is an inappropriate method to study sales interactions. Rather it implies that independent variables must be chosen that do not significantly interact with this aspect of artificiality. As discussed earlier, purely descriptive research would be problematic.

A special aspect of sales influence research that computers can facilitate is the bargaining process. A simulated salesperson may be programmed to respond to various types of consumer influence attempts with specific messages and concessions (e.g., Brucks and Schurr 1990; Schurr and Ozanne 1985). Computer-opponent bargaining eliminates the need for a human experimenter to trade messages with subjects and allows clean manipulations of bargaining strategy (i.e., one doesn't have to depend on a research assistant to follow instructions). Although the researcher must be careful about generalizing results to face-to-face bargaining situations, it is possible to conduct experiments on independent variables that would not be expected to interact with the "humanness of opponent" dimension.

Memory and Inference

Because computers make response time measurements easily obtainable, research on memory and inference is facilitated. Response time measures provide insight on the organization and content of product information in memory and the development of cognitive structures (Garder, Mitchell, and Russo 1978). They are commonly used to examine cognitive representations of category structures (Mervis and Rosch 1981). Since response times help differentiate responses based on memory from responses constructed on the spot, these measures may also be useful for distinguishing product judgments made during choice from product judgments made in response to experimenters' questions.

CONCLUSION

While the computer facilitates or makes possible a number of interesting research endeavors, it is not well-suited for all decision-making or judgment topics. For many research problems, the only major advantage of the computer is automation, which may not be important enough to offset the disadvantages - especially the time and expense of developing software for the task. It is recommended that the researcher carefully weigh the advantages and disadvantages before embarking on a computerized course.

REFERENCES

Bettman, James R. (1979), An Information Processing Theory of Consumer Choice, Reading, MA: Addison-Wesley.

Brucks, Merrie (1985), 'The Effects of Product Class Knowledge on Information Search Behavior," Journal of Consumer Research, 12 (June), 1-16.

Brucks, Merrie (1988), "Search Monitor: An Approach for Computer - Controlled Experiments Involving Consumer Information Search," Journal of Consumer Research, 15, (June) 117-121.

Brucks, Merrie and Dipankar Chakravarti (1986), "Knowledge of Interattribute Correlations in a Product Category: Effects on Consumer Choice Processes," Unpublished paper presented at the 1986 meeting of the Association for Consumer Research, Toronto, Ontario, Canada.

Brucks, Merrie and Andrew A. Mitchell (1981), "Knowledge Structures, Production Systems, and Decision Strategies," in Advances in Consumer Research, Vol. 3, ed. Kent Monroe, Ann Arbor: Association for Consumer Research, 750-757.

Brucks, Merrie and Paul H. Schurr (1990), "The Effects of Bargainable Attributes and Attribute Range Knowledge on Consumer Choice Processes," Journal of Consumer Research, forthcoming.

Chase, William G. and Herbert A. Simon (1973), "Perception in Chess," Cognitive Psychology, 4 (January), 55-81.

Chi, Michelene T. H., Robert Glaser, and Ernest Rees (1981), "Expertise in Problem Solving, Advances in the Psychology of Human Intelligence, ed. Sternberg, Hillsdale, NJ: Lawrence Erlbaum.

Drumwright, Minette E. (1986), 'The Effects of Prior Beliefs and Time Pressure on Covariation Assessments of Price-Quality Relationships for a Consumer Service," Unpublished dissertation, Graduate School of Business Administration, University of North Carolina, Chapel Hill, NC 27599-3490.

Gardner, Meryl, Andrew A. Mitchell, and J. Edward Russo (1978), "Chronometric Analysis: An Introduction and on Application to Low Involvement Perception of Advertisements," in Advances in Consumer Research, Vol. 5, ed. H. Hunt, Ann Arbor: Association for Consumer Research, 581-589.

Hoyer, Wayne D. and Jacob Jacoby (1983), "Three-Dimensional Information Acquisition: An Application to Contraceptive Decision Making," in Advances in Consumer Research, Vol. 10, eds. Richard P. Bagozzi and Alice M. Tybout, Ann Arbor, MI: Association for Consumer Research, 618-623.

Johnson, Eric J., John W. Payne, James R. Bettman, and David A. Schkade (1986), "Monitoring Information Acquisitions in Decision-Making: Experience with MouseLab, A Computer-Based Process Tracing System," paper presented at the Association for Consumer Research Annual Conference Toronto, Ontario, Canada.

Lynch, John G., Jr. (1982), "On the External Validity of Experiments in Consumer Research," Journal of Consumer Research, 9 (December), 225-239.

Mervis, Carolyn B. and Eleanor Rosch (1981), "Categorization of Natural Objects," Annual Review of Psychology, 32, 89-115.

Olshavsky, Richard W. and Dennis L. Rosen (1985), "Use of Product-Testing Organizations' Recommendations as a Strategy for Choice Simplification," Journal of Consumer Affairs, 19 (Summer), 118-139.

Ozanne, Julie L. (1988), "Keyword Recognition: A New Methodology for the Study of Information Seeking Behavior," in Advances in Consumer Research, Vol. 15, ed. Michael Houston, Provo, UT: Association for Consumer Research, 574579.

Ozanne, Julie L., Merrie Brucks, and Dhruv Grewal (1989), "The Influence of Information Discrepancy on Information Search Behavior During the Categorization Process," Working paper, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061.

Painton, Scott and James W. Gentry (1985), "Another Look at the Impact of Information Presentation Format," Journal of Consumer Research, 12 (September), 240-244.

Schurr, Paul H. and Julie L. Ozanne (1985), "Influences on Exchange Processes: Buyers' Preconceptions of a Seller's Trustworthiness and Bargaining Toughness," Journal of Consumer Research, 11 (March), 939-953.

Simon, Herbert (1967), "Motivational and Emotional Controls of Cognition," Psychological Review, 74, 29-39.

Rosen, Dermis L. and Richard W. Olshavsky (1987), "The Dual Role of Information Social Influence: Implications for Marketing Management," Journal of Business Research, 15 (April), 123-144.

Urbany, Joel E. (1986), "An Experimental Examination of the Economics of Information," Journal of Consumer Research, 13 (September), 257 -27 1.

Urbany, Joel E., William O. Bearden, and Dan C. Weilbaker (1988), 'The Effect of Plausible and Exaggerated Reference Prices on Consumer Perceptions and Price Search," Journal of Consumer Research, 15 (June), 95 - 110.

Wilkie, William L. and Peter R. Dickson (1985), "Consumer Information Search and Shopping Behavior," Working paper, University of Florida, Gainesville, FL 32611.

Zeithaml, Valarie and Merrie Brucks (1987), "Price as An Indictor of Quality Dimensions," Unpublished paper presented at the 1987 meeting of the Association for Consumer Research, Boston, MA.

----------------------------------------