Assessing Viewer Judgement of Advertisements and Vehicles: Scale Development and Validation

ABSTRACT - Lack of a common scale in marketing communications research hinders comparability among studies. This paper reports the development of a theory driven, bi-polar adjective scale, the Persuasive Discourse Inventory (PDI), which captures the fundamental elements of ads and/or media vehicles. If used, this inventory would foster greater comparability across studies.

Citation:

Tammi S. Feltham (1994) ,"Assessing Viewer Judgement of Advertisements and Vehicles: Scale Development and Validation", in NA - Advances in Consumer Research Volume 21, eds. Chris T. Allen and Deborah Roedder John, Provo, UT : Association for Consumer Research, Pages: 531-535.

Advances in Consumer Research Volume 21, 1994      Pages 531-535

ASSESSING VIEWER JUDGEMENT OF ADVERTISEMENTS AND VEHICLES: SCALE DEVELOPMENT AND VALIDATION

Tammi S. Feltham, Wilfrid Laurier University

[The author thanks S. J. Arnold, Queen's University, and the anonymous ACR reviewers for their insightful comments; and the Wilfrid Laurier University Post-Doctoral Research Fellowship for financial assistance.]

ABSTRACT -

Lack of a common scale in marketing communications research hinders comparability among studies. This paper reports the development of a theory driven, bi-polar adjective scale, the Persuasive Discourse Inventory (PDI), which captures the fundamental elements of ads and/or media vehicles. If used, this inventory would foster greater comparability across studies.

INTRODUCTION

Marketing communications researchers use various scales to describe the stimuli in their work. Examples include descriptions of ad content as "image" or "reason-why" (Aaker and Brown 1972), "humorous" or "non-humorous" (Murphy, Cunningham and Wilcox 1979), "thinking" or "feeling" (Golden and Johnson 1983), "cognitive" or "affective" (McClung, Park and Sauer 1985), "emotional" or "factual" (Liu and Stout 1987), and "happy" or "sad" (Goldberg and Gorn 1987). Examples of vehicle editorial environment descriptions include "prestigious" or "expert" (Aaker and Brown 1972), "drama" or "game show" (Wise, Brown, and Cox 1975), "documentary", "action/adventure", or "situation comedy" (Murphy et al. 1979), "cognitive" or "affective" (Park and McClung 1986), and "happy" or "sad" (Goldberg and Gorn 1987). Researchers and practitioners cannot arrive at reliable conclusions about the theory under investigation when the descriptions of the stimuli used vary from study to study. This non-uniformity leads to a situation where conclusions as to the effect of particular stimuli (for example: humorous ads) must be based on an assumption that the stimuli used in numerous investigations are comparable.

Diverse measures of stimuli in marketing communication research raise several problems. Firstly, when discussing the effect that a characteristic of a stimulus (i.e., humorous ads) has on a theoretical measure, it is difficult to know whether discrepant findings between studies are due to different measures or different behaviours. When different measures are used, how does a researcher know that the same phenomenon has been tested from one study to the next? Secondly, a problem may arise when measures employ single-item scales (and do not report reliability and validity testing results). Single-item scales lack richness of description, may not adequately capture the characteristics of the ads or vehicles, and may have low reliability. Thirdly, diverse measures obscure theoretical insights and explanations which might otherwise become apparent if all studies used a common method for describing and measuring ads and vehicles. Fourthly, current scales are descriptive rather than theoretically based. Thus a general, valid, multi-item, theory-based scale to measure ads and vehicles is needed.

To address these problems, this article develops and validates the Persuasive Discourse Inventory (PDI). This inventory contributes to marketing communication research by providing a standard classification structure for ad and vehicle stimuli which is comprehensive in nature. The PDI is based on the theory of Rhetoric (Aristotle 1984), wherein Aristotle discussed a tri-partite concept of a persuasive message. The three elements of the PDI, ethos, pathos, and logos, provide a common ground and basis of understanding across many disciplines. The PDI may provide comparability among studies, flexibility across diverse applications, quantitative measurability, and consistency to marketing communications research.

DEFINING PERSUASIVE DISCOURSE

In the Rhetoric, Aristotle stated: "Of the proofs which are furnished through the instrumentality of the speech itself, . . . there are three kinds, one residing in the character [ethos] of the speaker, the second in the feelings and emotions [pathos] produced in the audience, and the third, which is proof in its proper sense, logical, direct proof [logos], in the speech itself" (1867, Book I, Chapter II, ll. 1356^a1-4).

There is support for using Aristotle's writings as a basis for studying persuasive messages: "Aristotle discussed the processes of attitude change at considerable length. . . . His distinction, in the Rhetoric, between ethos, pathos, and logos as alternative processes for changing attitudes has been frequently elaborated in subsequent centuries" (McGuire 1969, p. 200). For example, extensive work on the three components was undertaken by the Yale group (Hovland, Janis and Kelley 1953; Hovland et al. 1957; Hovland and Weiss 1951).

The Persuasive Discourse perspective, as developed in this paper, is based on the theory of Rhetoric (Aristotle 1984). When applied to marketing stimuli, the theoretical components of the PDI (ethos, pathos, and logos) provide a thorough description of the constituent properties of the ads and vehicles used.

Ethos Appeals

In Aristotle's Rhetoric, ethos refers to "persuasive appeals that concentrate on the source rather than the message" (McGuire 1969, p. 200). The source effects literature has investigated issues which fall within the definition of ethos. Source characteristics research deals with the credibility and attractiveness of the spokesperson in a persuasive message. Attributions about the characteristics of a source can influence evaluations of an advertising message (positive correlation) (Chaiken 1979; Cooper and Croyle 1984; Percy 1983). Source credibility effects for vehicles have been found to be consistent with the effects found for ads (Andreoli and Worchel 1978; Wegner et al. 1981).

Pathos Appeals

Studies of advertising effects which have examined emotional or affective appeals fall within the definition of pathos. Several advertising and media studies have found an impact of mood states on the formation of consumer attitudes (Clark 1982; Isen 1984; Isen et al. 1978). Goldberg and Gorn (1987) defined vehicles as "happy" or "sad" and found that those watching the happy program rated the commercials as more effective, and had better recall than those watching the sad program. Research on the processes involved in emotional advertising has focused on the classification of emotion (Holbrook and O'Shaughnessy 1984; Stout and Leckenby 1986), and how emotion influences the persuasion process (Mitchell 1983, 1986a, 1986b; Mitchell and Olson 1981; Ray and Batra 1983).

Logos Appeals

Plato used logos in the sense of giving an account, and Aristotle used it as reason, or rationality (Peters 1967, p. 111). The philosophical dictionary terms used to define logos include account, reason, definition, and rational faculty (Peters 1967, p. 110). Logos appeals have been defined as reasoned discourse by Knepprath and Clevenger (1965, p. 152) who state that reasoned discourse "is a name given to the manner of putting words and ideas together in such a way that conclusions seem to follow." Cronkhite (1964, p. 16) defined a logical argument as the induction of audience perception of a relationship between concepts. McGuire (1969, p.202) considered as logical "those appeals which argue for the truth of a given belief by presenting evidence." In investigating rational ads, Pallak, Murroni, and Koch (1983, pp. 138-139) found that positive "product/message-oriented thoughts" were predictive of product evaluations. A logos appeal, whether in a vehicle or an ad, appears to provide evidence or information about a concept (i.e., a product) from which a consumer can form and evaluate beliefs.

Separability of Persuasive Discourse Constructs

While ethos appeals concentrate on the source of the message, pathos and logos appeals persuade by focusing attention on the message content (Rosenthal 1966). While marketing studies commonly examine the previously defined concepts of pathos and logos as opposite ends of a continuum (Goldberg & Gorn 1987; Golden & Johnson 1983; Holbrook 1978; McClung et al. 1985; Park & McClung 1986), there is precedent for treating these constructs separately. These factors most likely do not represent two ends of a continuum; they are two distinct dimensions (Knepprath and Clevenger 1965; McGuire 1969; Percy 1983; Pallak et al. 1983).

There is also precedent for treating logos and ethos (as defined in the PDI) as separable. Pallak et al. (1983) found that source effects due to the expertness or attractiveness of the communicator had no significant effect on the evaluation of the product when the message was rationally based. It seems that "students in the rational-message conditions thought about the content of the message," as opposed to thinking about the communicator (p. 135). They concluded that the rational advertisements were processed systematically, based on the message content.

PERSUASIVE DISCOURSE SCALE DEVELOPMENT

For complete results and tables, contact the author.

Stimuli selection

A three part, bi-polar adjective, Persuasive Discourse scale was developed based on the above definitions of ethos, pathos, and logos. Television was chosen as the medium for scale development and validation, although it is assumed that the final PDI will apply to radio and print studies (confirmation awaits further testing). Multiple scenes from a single movie were used for the program stimuli because of the wide range of ethos, pathos, and logos found in dramatic presentations (while attempting to decrease the variability of characters and settings across stimuli conditions, i.e., hold extraneous characteristics constant). Commercials considered for the study were chosen from the product categories of wrist watches, candy, photography equipment, coffee and pain relievers. The criterion for choosing the commercials was that the advertised products be of interest to an adult population. In addition, no commercial had been aired in the previous five years and no commercial was distinguished by any unusual exposure history.

Item generation

Once a pool of commercials and film clips was gathered, scale items were generated. The objective was to develop a valid and reliable scale for differentiating vehicles and advertisements on the dimensions of ethos, pathos, and logos. Items were generated from a lexicon of philosophical terms (Peters 1967), an encyclopedia of philosophy (Kerferd 1967), speech communication literature (Cronkhite 1964; Knepprath and Clevenger 1965; Rosenthal 1966), and consultation with colleagues for each of the ethos, pathos, and logos dimensions.

Data collection

A questionnaire was developed from the item pool using seven point bi-polar adjective scales. The resulting questionnaire was administered to a convenience sample of nineteen adult volunteers (mean: 42.3 years, median: 35 years, range: 24 to 72 years). Volunteers were told they would see a series of movie clips and television commercials which they would be asked to rate on the basis of the scale items listed on the questionnaire. Procedurally, they viewed a video stimulus, the tape was paused, and they then filled out the scale items for that stimulus. The next clip or ad was played, the tape paused, and so on through sixteen stimuli (five movie clips, then eleven ads).

Measure Purification

The pool of seventeen persuasive discourse scale items were subjected to an internal validity analysis, based on performance across the sixteen media and ad stimuli. Individual scale items were examined for correlation to total score for each related component variable, i.e., E1, E2, E3, E4, E5 to Ethos (summation of ethos scores on items E1 through E5); L1, L2, L3, L4, L5, L6 to Logos (same procedure as Ethos); and P1, P2, P3, P4, P5, P6 to Pathos (same procedure as Ethos). The objective was to establish that each item contributed to the same component concept, "thereby establishing the homogeneity or internal consistency of the component variables" (Seymour and Lessne 1984, p. 813). The average correlation matrix revealed that scale items had item-to-total correlation coefficients ranging from 0.63 to 0.91.

While the subscales of the PDI (Ethos, Pathos, Logos) measure three separate and distinct aspects of the test stimuli as shown by item-to-total correlations, it was discovered that the ethos of the ads tended to follow the logos levels. The nature of advertising would support a positive correlation between Logos and Ethos; if a person/message is seen as convincing, informative, and logical, then it is also seen as believable, credible, and trustworthy.

An examination of the PDI scale scores for each program and ad revealed that each subscale component (Ethos, Pathos, and Logos) differentiated among the sixteen ad and program test stimuli. The scores for Ethos were distributed fairly evenly across the range of the Ethos subscale from 14.7 to 29.5 (possible range 5 to 35); i.e., the test stimuli were perceived to vary in terms of their ethos content. Likewise, the scores for the Pathos subscale ranged from 13.2 to 33.4 (possible range 7 to 49), and the Logos subscale ranged from 16.7 to 33.2 (possible range 5 to 35).

The recommended measure of internal consistency of a set of items is provided by coefficient alpha. According to Churchill (1979, p. 68) it "absolutely should be the first measure one calculates to assess the quality of the instrument. . . . Thus, a low coefficient alpha indicates the sample of items performs poorly in capturing the construct which motivated the measure". Reliability analysis (coefficient alpha) was run for Ethos, Logos, and Pathos for each of the sixteen stimuli. The average coefficient alphas over the sixteen stimuli were 0.8903 for Ethos (range: O.6092 to 0.9819), 0.8593 for Logos (range: 0.3994 to 0.9819), and 0.8215 for Pathos (range: 0.5915 to 0.9684). While Nunnally (1967, p. 226) suggests that increasing reliabilities beyond 0.80 is probably wasteful, using the correlation matrix and coefficient alpha scores, the PDI scale items were revised by dropping a poorly performing Logos item and replacing several poorly performing Pathos items with new items. Additional data was then collected.

EXHIBIT 1

PERSUASIVE DISCOURSE INVENTORY QUESTIONNAIRE

Reliability assessment with new data

According to Churchill (1979), the next step in assessing reliability of a measure is to collect additional data because "coefficient alpha does not adequately estimate, . . . errors caused by factors external to the instrument, such as differences in testing situations and respondents over time" (p. 70). The second test consisted of seven commercials and three movie clips. Procedures were the same as in the first data collection period. The second test was conducted among volunteers from four introductory marketing classes (n= 25, 19, 15, and 20 respectively). To check for scale item order effects, the instrument items were ordered in two ways and both versions distributed equally in each session. An example of the instrument is found in Exhibit 1. The purpose of this test was to replicate and refine the scale items. The average correlation matrix (to examine item-to-total, inter-item, and cross-correlations) was calculated. Reliability analysis (coefficient alpha) was conducted on the three scales of Ethos, Pathos, and Logos. Examination of the two forms of the instrument for each of the sessions revealed no significant differences in mean responses due to scale item presentation order. Scale item order was, therefore, ignored in further analyses.

Examination of the average correlation matrix, summing over nine vehicles and ads (commercial one served to introduce participants to the task), revealed that the adjustments to the PDI scale items improved the subscale inter-item correlations, while retaining low cross-correlations among the Ethos, Pathos, and Logos scales. The average correlation of items-to-total for the Pathos scale was .79. The Ethos and Logos scale items' inter-item correlations fell from those recorded in earlier testing. However, the cross-correlations between Ethos and Logos also declined (desirable). The average correlation of items-to-total for the Ethos scale was .78. The average correlation of items-to-total for the Logos scale was .74.

To further examine the internal reliability of the Ethos, Pathos, and Logos subscales, coefficient alpha was calculated. Reliability scores averaged across nine stimuli were .83 for the Ethos scale (range: 0.6736 to 0.9167); .79 for the Logos scale (range: 0.5903 to 0.8893); and .89 for the Pathos scale (0.8173 to 0.9671). Coefficient alpha scores fell slightly for the Ethos and Logos subscales and improved for the Pathos subscale (note the previous adjustment of items and new sample of ads/programs rated). In general, the reliabilities are acceptable and stable across situations.

Additional replications were conducted with volunteers from an MBA marketing class (n = 15) and volunteers from a local choir (n = 7). These sessions supported the findings of the four introductory marketing sessions for a population with an older age range.

The final format of the PDI contains seventeen items; five for Ethos, five for Logos, and seven for Pathos (Exhibit 2).

CONCLUSIONS

This paper develops and validates a 7 point bi-polar adjective scale, the PDI, based on Aristotle's Rhetoric. The use of this multi-item, theoretically based scale allows for greater comparability, providing the potential for greater insight both within and across studies and for a common measurement standard for media research. Field tests in upcoming research projects are planned to further validate the scale.

In particular, these tests will examine the convergent and discriminant validity of the PDI scale to ensure that the scale is validly measuring the theoretical components of a persuasive communication. The development of the PDI to this point leaves certain questions unanswered. For example, does the PDI capture figurative elements of language in ads and programs, and executional dimensions such as vividness?

EXHIBIT 2

PERSUASIVE DISCOURSE INVENTORY

While each of the three subscales, Ethos, Pathos, and Logos can be used independently, the greatest potential benefit lies in using the entire PDI scale to choose the ads and programs used in research. With a more comprehensive characterization available, regardless of the theory being tested or the phenomenon under investigation, additional insights may materialize and some of the data limitations of current research may be avoided.

The PDI can provide additional control over ad and program stimuli used in many consumer studies. For example, the researcher could use the PDI to select ads with the characteristic of interest, high pathos, while measuring and controlling the levels of ethos and logos. A specific area which lends itself well to the use of the PDI is the influence of program context on advertising effectiveness. In this area, the issue of defining congruity between an ad and a program is important; the PDI would provide a standard definitional base for defining/matching congruent ads and programs.

REFERENCES

Aaker, David A. and Phillip K. Brown (1972), "Evaluating Vehicle Source Effects," Journal of Advertising Research, 12 (August), 11-16.

Andreoli, Virginia and Stephen Worchel (1978), "Effects of Media, Communicator, and Message Position on Attitude Change," Public Opinion Quarterly, 42 (Spring), 59-70.

Aristotle (1867), Rhetoric, trans. E. M. Cope, in An Introduction to Aristotle's Rhetoric, Cambridge: Macmillan and Co.

Aristotle (1984), Rhetoric, trans. W. Rhys Roberts, in The Complete Works of Aristotle: The Revised Oxford Translation, ed. Jonathan Barnes. Bollingen Series LXXI, 2. Princeton, New Jersey: Princeton University Press.

Chaiken, Shelly (1979), "Communicator Physical Attractiveness and Persuasion," Journal of Personality and Social Psychology, 37, 1387-1397.

Churchill, Gilbert A. Jr. (1979), "A Paradigm for Developing Better Measures of Marketing Constructs," Journal of Marketing Research, XVI (February), 64-73.

Clark, Margaret (1982), "A Role for Arousal in the Link Between Feeling States, Judgments, and Behavior," in Affect and Cognition, eds. Margaret Clark and Susan Fiske. Hillsdale, NJ: Erlbaum, 263-289.

Cooper, Joel and Robert T. Croyle (1984), "Attitudes and Attitude Change," Annual Review of Psychology, 35, 395-426.

Cronkhite, Gary Lynn (1964), "Logic, Emotion, and the Paradigm of Persuasion," Quarterly Journal of Speech, L, 13-18.

Goldberg, Marvin E. and Gerald J. Gorn (1987), "Happy and Sad TV Programs: How They Affect Reactions to Commercials," Journal of Consumer Research, 14 (December), 387-403.

Golden, Linda L. and Keren A. Johnson (1983), "The Impact of Sensory Preference and Thinking Versus Feeling Appeals on Advertising Effectiveness," in Advances in Consumer Research, Volume X, eds. Richard P. Bagozzi and Alice M. Tybout. Ann Arbor, MI: Association for Consumer Research, 203-208.

Holbrook, Morris B. (1978), "Beyond Attitude Structure: Toward the Informational Determinants of Attitude," Journal of Marketing Research, XV, 545-556.

Holbrook, Morris B. and John O'Shaughnessy (1984), "The Role of Emotion in Advertising," Psychology & Marketing, 1 (Summer), 45-64.

Hovland, Carl I., Irving L. Janis, and Harold H. Kelley (1953), Communication and Persuasion, New Haven: Yale University Press.

Hovland, Carl I., Wallace Mandell, Enid H. Campbell, Timothy Brock, Abraham S. Luchins, Arthur R. Cohen, William J. McGuire, Irving L. Janis, Rosalind L. Feierabend, and Norman H. Anderson (1957), The Order of Presentation in Persuasion, New Haven: Yale University Press.

Hovland, Carl I. and Walter Weiss (1951), "The Influence of Source Credibility on Communication Effectiveness," Public Opinion Quarterly, 15 (Winter), 635-650.

Isen, Alice M. (1984), "The Influence of Positive Affect on Decision Making and Cognitive Organization," in Advances in Consumer Research: Volume XI, ed. Thomas C. Kinnear. Provo, UT: Association for Consumer Research, 534-537.

Isen, Alice M., Margaret Clark, Thomas E. Shalker, and Lynn Karp (1978), "Affect, Accessibility of Material in Memory and Behavior: A Cognitive Loop?" Journal of Personality and Social Psychology, 36, 1-12.

Kerferd, G. B. (1967), "Logos," The Encyclopedia of Philosophy, Volume 5, ed. Paul Edwards. New York: Macmillan Publishing Co., Inc. & The Free Press.

Knepprath, Eugene and Theodore Jr. Clevenger (1965), "Reasoned Discourse and Motive Appeals in Selected Political Speeches," Quarterly Journal of Speech, LI, 152-156.

Liu, Scott S. and Patricia A. Stout (1987), "Effects of Message Modality and Appeal on Advertising Acceptance," Psychology & Marketing, 4(3), 167-187.

McClung, Gordon W., C. Whan Park, and William J. Sauer (1985), "Viewer Processing of Commercial Messages: Context and Involvement," in Advances in Consumer Research, Volume XII, eds. Elizabeth C. Hirschman and Morris B. Holbrook. Provo, UT: Association for Consumer Research, 351-355.

McGuire, William J. (1969), "The Nature of Attitude and Attitude Change," The Individual in a Social Context, Vol. III of The Handbook of Social Psychology, eds. Gardner Lindzey and Elliot Aronson, 2nd ed. Reading, MA: Addison-Wesley, 136-314.

Mitchell, Andrew A. (1983), "The Effects of Visual and Emotional Advertising: An Information-Processing Approach," Advertising and Consumer Psychology, eds. Larry Percy and Arch G. Woodside. Lexington, Massachusetts: Lexington Books, D.C. Heath and Company, 197-217.

Mitchell, Andrew A. (1986a), "Some Issues Surrounding Research on the Effects of 'Feeling Advertisements'," Advances in Consumer Research, Vol. XIII, ed. Richard J. Lutz. Provo, UT: Association for Consumer Research, 623-628.

Mitchell, Andrew A. (1986b), "The Effect of Verbal and Visual Components of Advertisements on Brand Attitudes and Attitude Toward the Advertisement," Journal of Consumer Research, 13 (June), 12-24.

Mitchell, Andrew A. and Jerry C. Olson (1981), " Are Product Attribute Beliefs the Only Mediator of Advertising Effects on Brand Attitude?" Journal of Marketing Research, 18, 318-332.

Murphy, John H., Isabella C.M. Cunningham, and Gary B. Wilcox (1979), "The Impact of Program Environment on Recall of Humorous Television Commercials," Journal of Advertising, 8 (2), 17-21.

Nunnally, Jum C. (1967), Psychometric Theory, New York: McGraw-Hill Book Company.

Pallak, Suzanne R., Enid Murroni, and Juliann Koch (1983), "Communicator Attractiveness and Expertise, Emotional Versus Rational Appeals, and Persuasion: A Heuristic Versus Systematic Processing Interpretation," Social Cognition, 2, 122-141.

Park, C. Whan and Gordon W. McClung (1986), "The Effect Of TV Program Involvement On Involvement With Commercials," in Advances in Consumer Research, Volume XIII, ed. Richard J. Lutz. Provo, UT: Association for Consumer Research, 544-548.

Percy, Larry (1983), "A Review of the Effect of Specific Advertising Elements upon Overall Communication Response," in Current Issues & Research in Advertising, 6, 77-118.

Peters, F. E. (1967), Greek Philosophical Terms: A Historical Lexicon, New York: New York University Press.

Ray, Michael L. and Rajeev Batra (1983), "Emotion and Persuasion in Advertising: What We Do and Don't Know About Affect," Advances in Consumer Research: Vol. X, eds. Richard P. Bagozzi and Alice M. Tybout. Ann Arbor, MI: Association for Consumer Research, 543-547.

Rosenthal, P. I. (1966), "Concept of Ethos and the Structure of Persuasive Speech," Speech Monographs, 33, 114-126.

Seymour, Daniel and Greg Lessne (1984), "Spousal Conflict Arousal: Scale Development," Journal of Consumer Research, 11 (December), 810-821.

Stout, Patricia A. and John D. Leckenby (1986), "Measuring Emotional Response to Advertising," Journal of Advertising, 15, No. 4, 35-42.

Wegner, Daniel M., Richard Wenzlaff, R. Michael Kerker, and Ann E. Beattie (1981), "Incrimination Through Innuendo: Can Media Questions Become Public Answers?" Journal of Personality and Social Psychology, 40, 822-832.

Wise, Gordon L., Herbert E. Brown, and Myron K. Cox (1975), "The Effect of Program Type and Other Variables in Reaching the Daytime Television Viewer with Advertising Messages," Journal of Advertising, 4 (3), 41-46.

----------------------------------------