Special Session Summary Novel Experimental Methods: Opportunities and Challenges

Citation:

Cristel A. Russell and Christopher P. Puto (1999) ,"Special Session Summary Novel Experimental Methods: Opportunities and Challenges", in NA - Advances in Consumer Research Volume 26, eds. Eric J. Arnould and Linda M. Scott, Provo, UT : Association for Consumer Research, Pages: 599-600.

Advances in Consumer Research Volume 26, 1999      Pages 599-600

SPECIAL SESSION SUMMARY

NOVEL EXPERIMENTAL METHODS: OPPORTUNITIES AND CHALLENGES

Cristel A. Russell, The University of Arizona

Christopher P. Puto, Georgetown University

This session was designed in accordance with the 1998 ACR Conference theme, "Dialogue, Difference, and Delight." The session’s focus was on non-traditional experimental methodologies for consumer behavior research. Specifically, three novel, state-of-the-art experimental methodologies were presented, and the opportunities and challenges associated with each were discussed.

Shortcomings in the experimental method are frequently noted in the consumer behavior discipline (e.g., Calder, Phillips, and Tybout, 1981, 1982, Lynch 1982). However, the debate over priorities among validity types (see Cook and Shadish, 1994) may soon become moot as recent technological and/or creative developments in experimental methods suggest that settings commonly labeled "artificial" may not be so artificial anymore (see Kardes 1996).

This session explored some of the most novel experimental methodologies in consumer behavior research. First, Russell, Puto, and Lang’s paper introduced the use of theatre as an original but valid experimental setting for studying cultural phenomena. Kosslyn, Braun, and Zaltman’s paper presented the application of state-of-the-art cognitive neuroscience technologies and methodologies in the study of consumer behavior. Finally, Brown`s paper explored novel uses of interactive computer-based technologies in research on consumer preference formation and discussed the analytical risks and opportunities associated with them.

Each of the three presentations provided a new perspective for experimental design and added to the set of methodological choices available to consumer researchers. In addition, these presentations all transcended traditional disciplinary boundaries by integrating methodologies and technologies from different fields.

To further broaden the scope of the session, Elizabeth Hirschman, from Rutgers University, integrated these diverse perspectives and opened the discussion about these new horizons for experimental research.

 

"EXPERIMENTAL WORK ON STAGE: THE NEW THEATRE METHODOLOGY FOR EXPLORING CULTURAL PHENOMENA."

Cristel Russell, University of Arizona

Christopher Puto, Georgetown University

William Lang, University of Arizona

In this presentation, we provided a thorough description of a new theatre methodology, presented its advantages in terms of experimental design, and discussed its potential uses in consumer behavior and marketing research. The theatre methodology relies on the use of a theater environment as the experimental setting for the presentation of stimuli. The medium for delivery of the various treatments was an original play written by one of the authors. The methodology was developed to test a theory which proposes that products gain meaning from their placement within cultural environments (Russell, 1998).

The main motivation for using a theatrical setting was to create a culturally rich environment while maintaining a high level of experimental control. The nature of the product placement theory to be tested required that different levels of visual appearances, verbal mentions, and connection to the story line be manipulated. Using "real" stimuli (e.g., existing television shows) would reduce the flexibility of the design by limiting the number of experimental conditions to the available programs. This, in turn, would seriously compromise internal validity. Hence, by writing our own script and producing our own show, we were able to reproduce the characteristics of real television shows while creating the exact experimental conditions needed to test our model.

To illustrate the potential of the theatre methodology, a clip of the live performance was shown and results from a post-performance survey and focus groups conducted with members of the audience were discussed. The theatrical experience generated strong levels of involvement as well as very positive attitudes toward the play.

This new methodology resulted from efforts to extend this research program across traditional disciplinary boundaries. The close collaboration of the media and theatre arts departments allowed us to increase the quality and believability of the plot, facilitated our access to professional actors, and assisted with the logistics of the play reading and videotaping sessions.

We view the theatre methodology as an attractive alternative for experimental design, one able to enhance both the internal and the external validity of traditional "laboratory" experiments. The theatre methodology situates itself at the intersection of two research paradigms, combining aims traditionally associated with the interpretive paradigm with the experimental method of the positivist paradigm. Because of its ability to study cultural phenomena within a controlled environment, the potential applications of this new methodology in consumer behavior research are numerous.

 

"APPLICATIONS OF COGNITIVE NEUROSCIENCE IN CONSUMER BEHAVIOR"

Stephen Kosslyn, Harvard University

Kathryn Braun, Harvard University

Gerald Zaltman, Harvard University

Recent advances in understanding brain structure and functioing and improvements in physiological measurement technologies are increasing the opportunities to gain deeper insight into both tacit and explicit knowledge consumers hold about a variety of consumption situations and their responses to various marketing stimuli. These advances include, but are not limited to, the use of electronic scanning methods such as positron emission tomography (PET) to monitor brain functioning. An example was presented involving the use of PET scanning to evaluate consumer responses to alternative retail environments. The promise and difficulties of such techniques were discussed along with other implications arising from their use.

 

"INSIDE THE DATA VOLCANO: RISKS AND OPPORTUNITIES OF TECHNOLOGY-DRIVEN METHODOLOGICAL INNOVATIONS"

Christina Brown, University of Michigan

This presentation 1) demonstrated some novel uses of interactive computer-based technologies in research on consumer preference formation, 2) provided a framework for deciding when and how to use them, and discussed the analytical risks and opportunities imposed by the remarkable flow of data they produce, and 3) demonstrated the use of analytical tools, such as hierarchical linear modeling, to assess how information presentation format affects preference formation. (For example, one treatment might generate slow steady improvement in attitude whereas another may generate an immediate improvement followed by a decline. How would we test the differences between these two?) These methods were demonstrated and compared to more traditional methods.

Some fields of investigation have been stymied by natural limitations of the methodologies used to understand them. For example, an experimenter interested in the persuasiveness of an ad may be limited to a single exposure, with pre- and post-tests of product evaluation. However, the value of the ad might accrue only over time. Thus, although the phenomenon develops over time, our ability to manage a lengthy experiment limits the way in which we study the phenomenon. A researcher might also prefer a simple pre-exposure-post test paradigm to the analytical challenges imposed by a lengthier sequence of exposures. Novel technologically-driven methodologies, such as computer-based interactive experiments, offer new opportunities to present, manipulate, and record reactions to a lengthy sequence of stimuli. For example, during a single computer-simulated real estate tour, we may view a large number of apartments in a single brief session at a terminal. Thus, instead of just a few ratings of apartments, an experimenter may obtain a lengthy sequence of ratings. Other measures, such as response times, suddenly become relatively easy to collect. Real-time reaction data to experiential stimuli (for example, continuous ratings over the course of a song or commercial) vastly increase the amount of data collected from a single experiment. This "data explosion" offers new challenges to analyze the data correctly and to extract its full value.

This presentation used results of several experiments to demonstrate two principles that should underlie the choice to "tech up" one’s experiments. First, a paradigm should naturally flow from the theoretical questions at hand. For example, the author’s interest in processes of preference formation and change naturally implies the collection of a long sequence of data on individuals. Since consumers have some control over the environment in which they learn about their own preferences, this implies interactivity: the way in which the experiment looks in later stages must be able to change depending upon the responses of the consumer in the first stages of the sequence. Computer-based experiments vastly facilitate this process. This interactivity was illustrated using a shopping simulation program developed by the author. Secondly, an experimenter must be able to make good use of the remarkable flow of data that such experiments generate. The presentation demonstrated use of time-series and hierarchical linear analysis to analyze the difference in preference foration and attitude "growth curves"-changes in attitude over time.

REFERENCES

Calder, B. J., L. W. Phillips, and A. M. Tybout (1981) "Designing Research for Application," Journal of Consumer Research, 8, 197-207

Calder, B. J., L. W. Phillips, and A. M. Tybout (1982) "The Concept of External Validity," Journal of Consumer Research, 9, 240-244

Cook, Thomas D. and William R. Shadish (1994) "Social Experiments: Some Developments over the Past Fifteen Years," Annual Review of Psychology, 45:545-580

Kardes, Frank R. (1996) "In Defense of Experimental Consumer Psychology," Journal of Consumer Psychology, 5(3), 279-296

Lynch, J. (1982) "On the External Validity of Experiments in Consumer Research," Journal of Consumer Research, 9, 225-239

Russell, Cristel A. (1998) "Toward A Framework of Product Placement: Theoretical Propositions," in Joseph W. Alba and J. Wesley Hutchinson (eds.), Advances in Consumer Research, Vol. XXV, 357-362.

----------------------------------------