The Consumer Versus the Judge: an Empirical Comparison of Approaches to Content Analysis in Cross-Cultural Advertising Research

EXTENDED ABSTRACT - Recent interest in cross-cultural advertising research has led to several empirical studies that examine similarities and differences in advertising content between various countries. Cross-cultural studies of this kind typically use content analysis as the primary, if not only, method for comparing ads. Whereas content analysis was originally devised to quantify qualitative data through the capture of the presence, or frequency, of a word or object, it is more widely applied today to capture meaning. Such usage suggests that the selection of judges together with the type of content to be coded might have a significant impact on validity regardless of the care with which coding categories are devised and coder instructions prepared. For example, the language background (i.e. monolingual or bilingual) of the judges selected to code the material may influence the reliability and validity of the findings. Additionally, the type of content (i.e., object or narrative) selected may have an effect on the level of implicit versus explicit interpretation that is required from the judge.

Citation:

Dawn Lerman and Michael Callow (2003) ,"The Consumer Versus the Judge: an Empirical Comparison of Approaches to Content Analysis in Cross-Cultural Advertising Research", in NA - Advances in Consumer Research Volume 30, eds. Punam Anand Keller and Dennis W. Rook, Valdosta, GA : Association for Consumer Research, Pages: 230-231.

Advances in Consumer Research Volume 30, 2003     Pages 230-231

THE CONSUMER VERSUS THE JUDGE: AN EMPIRICAL COMPARISON OF APPROACHES TO CONTENT ANALYSIS IN CROSS-CULTURAL ADVERTISING RESEARCH

Dawn Lerman, Fordham University

Michael Callow, Morgan State University

EXTENDED ABSTRACT -

Recent interest in cross-cultural advertising research has led to several empirical studies that examine similarities and differences in advertising content between various countries. Cross-cultural studies of this kind typically use content analysis as the primary, if not only, method for comparing ads. Whereas content analysis was originally devised to quantify qualitative data through the capture of the presence, or frequency, of a word or object, it is more widely applied today to capture meaning. Such usage suggests that the selection of judges together with the type of content to be coded might have a significant impact on validity regardless of the care with which coding categories are devised and coder instructions prepared. For example, the language background (i.e. monolingual or bilingual) of the judges selected to code the material may influence the reliability and validity of the findings. Additionally, the type of content (i.e., object or narrative) selected may have an effect on the level of implicit versus explicit interpretation that is required from the judge.

One option is for researchers to select judges that appear similar to the target audience and are therefore presumably able and qualified to correctly interpret the ads and assign valid codes. This typically means using two sets of monolingual judges each to code ads from their own country. Since these judges presumably have the linguistic and cultural knowledge required to properly classify the ads under study, this approach represents an attempt to increase interjudge reliability, category reliability, and validity within each country analysis. However, the use of monolingual judges poses a methodological problem when researchers wish to compare advertising content between countries. More specifically, interjudge reliability cannot be calculated across the various stimuli collected from the countries under study unless there is an overlap in material which is not the case.

In recognition of this limitation, some researchers have used a bilingual judge in conjunction with monolingual judges, or alternatively, a single set of bilingual judges. Unfortunately, interjudge reliability between bilingual and monolingual judges tends to fall below the recommended guideline. Bilingual judges, however, tend to achieve higher interjudge reliability rates thus allowing the researcher to determine whether the codes have been applied to the ads from both countries in a consistent fashion. However, consistent application of a coding scheme does not guarantee validity nor does it guarantee that the code is being used in a manner that is relevant to the hypotheses under question. In other words, the judges must be cultural translators as well as language translators.

One remedy to these methodological problems suggested by Lerman and Callow (1999) is the development of narrative texts from native consumers for each ad under study. The purpose of this stage is to convert any implicit messages in the ads to explicit information. Once this cultural interpretation has taken place, judges are then used to categorize the narratives. Since the material has been converted from an advertising format comprising both visual and verbal elements into more explicit textual narratives, the judges can rely on language skills instead of interpretative skills to classify the material.

These various approaches to content analysis were tested in a study comparing ads from Spain and the United States. Five pairs of judges, two bilingual and three monolingual, coded either the ads directly or narratives of the ads. Ten ads for hard liquor and ten ads for cars were selected from each country. All ads appeared in either the American magazine GQ or the Spanish magazine Quo. Ten consumers in each country who fit the demographic profile of the magazine readership (male between the ages of 22 and 35) and thus are presumably in the target market for the ads served as ad interpreters and provided the narratives to be coded. The coding scheme consisted of nine codes: traditional, modern, productivity, enjoyment, independence, status, affiliation, family, and morality. These codes represent a subset of values that Pollay (1983) identified as expressed in advertising and recommends for use as a coding scheme in advertising research.

A comparison of both interjudge reliability rates and substantive results based on a series of chi-square analyses indicates the superiority of narrative coding over ad coding in cross-cultural advertising research. The results also offer evidence that monolingual narrative coding is preferable to bilingual narrative coding. These findings call into question the results of numerous cross-cultural advertising studies. More specifically, they suggest that their findings are, at least in part, a function of the approach chosen by the researchers. The fact that researchers have used different approaches to content analysis introduces further confusion since the direction of the error may vary from study to study. Given these issues, the paper makes a number of recommendations for increasing the methodological rigor of content analysis for cross-cultural advertising research and offers some direction for making sense of prior research results in light of methodological concerns.

REFERENCES

Aaker, Jennifer and Patti Williams (1998), "Empathy versus Pride: The Influence of Emotional Appeals across Cultures," Journal of Consumer Research, 25 (December), 241-261.

Biswas, Abhijit, Janeen E. Olsen, and Valerie Carlet (1992), "A Comparison of Print Advertisements from the United States and France," Journal of Advertising, 21 (4), 73-81.

Briley, Donnel A., Michael W. Morris, and Itamar Simonson (2000), "Reasons as Carriers of Culture: Dynamic versus Dispositional Models of Cultural Influence on Decision Making," Journal of Consumer Research, 27 (September), 157-178.

Cho, Bongjin, Up Kwon, James W. Gentry, Sunkyu Jun and Frederic Kropp (1990, "Cultural Values Reflected in Theme and Execution: A Comparative Study of U.S. and Korean Television Commercials," Journal of Advertising, 28 (4), 59-73.

Cook, Guy (1992), The Discourse of Advertising. Routledge: New York.

Cutler, Bob D., and Rajshekhar G. Javagli (1992), "A Cross-Cultural Analysis of the Visual Components of Print Advertising: The United States and the European Community," Journal of Advertising Research, 32 (1), 71-80.

Eco, Umberto (1979), A Theory of Semiotics, Bloomington: Indiana University Press.

Ford, John B., Patricia Kramer Voli, Earl D. Honeycutt Jr., and Susan L. Casey (1998), "Gender Role Portrayals in Japanese Advertising: A Magazine Content Analysis," Journal of Advertising, 27 (1), 113-124.

Fraser, Bruce (1993), "The Interpretation of Novel Metaphors," in Andrew Ortony, ed., Metaphor and Thought. Cambridge University Press, 329-341.

Graham, John L., Michael A. Kamins, and Djoko S. Oetomo (1993), "Content Analysis of German and Japanese Advertising in Print Media from Indonesia, Spain, and the United States," Journal of Advertising, 22 (2), 5-15.

Hall, Edward T. (1981), Beyond Culture, Doubleday: New York.

Hall, Edward T. (1959), The Silent Language, Doubleday: New York.

Kassarjian, Harold H. (1977), "Content Analysis in Consumer Research," Journal of Consumer Research, 4 (June), 8-18.

Kolbe, Richard H. and Melissa S. Burnett (1991), "Content-Analysis Research: An Examination of Applications with Directives for Improving Research Reliability and Objectivity," Journal of Consumer Research, 18 (September), 243-250.

Krippendorff, Klaus (1980), Content Analysis: An Introduction to its Methodology, Beverly Hills, CA: Sage.

Lerman, Dawn B. and Michael A. Callow (1999), "Content Analysis in Cross-Cultural Advertising Research: Limitations and Recommendations," European Advances in Consumer Research, Bernard Dubois, Tina M. Lowrey, L. J. Shrum, and Marc Vanjuele, eds., Provo, UT: Association for Consumer Research, 4, 13-16.

Malhotra, Naresh K., James Agarwal, and Mark Peterson (1996), "Methodological Issues in Cross-Cultural Marketing Research," International Marketing Review, 13 (5), 7-43.

Maynard, Michael L., and Charles R. Taylor (1999), "Girlish Images Across Cultures: Analyzing Japanese versus U.S. Seventeen Magazine Ads," Journal of Advertising, 28 (1), 39-48.

McQuarrie, Edward F. and David Glen Mick (1999), "Visual Rhetoric in Advertising: Text-Interpretive, Experimental, and Reader-Response Analyses," Journal of Consumer Research, 26 (June), 37-54.

Mick, David Glen and Laura G. Politi (1989), "Consumer Interpretations of Advertising Imagery: A Visit to the Hell of Connotation," in Interpretive Consumer Research, ed. Elizabeth Hirschman, Provo, UT: Association of Consumer Research, 85-96.

Phillips, Barbara J. (1997), "Thinking Into It: Consumer Interpretation of Complex Advertising Images," Journal of Advertising, 26 (Summer), 77-87.

Pollay, Richard (1983), "Measuring the Cultural Values Manifest in Advertising," in Current Issues and Research in Advertising, eds. Leigh Martin and Claude R. Martin Jr., Ann Arbor: University of Michigan, 72-92.

Scott, Linda (194), "Images in Advertising: The Need for a Theory of Visual Rhetoric," Journal of Consumer Research, 21 (September), 252-271.

Schmitt, Bernd H. and Shi Zhang (1998), "Language Structure and Categorization: A Study of Classifiers in Consumer Cognition, Judgment, and Choice," Journal of Consumer Research, 25 (September), 108-122.

Sin, Leo Y. M., Gordon W. H. Cheung, and Ruby Lee (1999), "Methodology in Cross-Cultural Consumer Research: A Review and Critical Assessment," Journal of International Consumer Marketing, 11 (4), 75-96.

Wiles, Charles R., Judith A. Wiles, and Anders Tjernlund (1996), "The Ideology of Advertising: The United States and Sweden," Journal of Advertising Research, 36 (3), 57-66.

Zhang, Yong, and Betsy D. Gelb (1996), "Matching Advertising Appeals to Culture: The Influence of Products’ Use Conditions," Journal of Advertising, 25(3), Fall, 29-46.

----------------------------------------