Search For a Relationship Between Viewer Responses to the Creative Aspects of Televised Messages and Behavioral Intention

ABSTRACT - Subjects' responses to creative stimulus items for televised messages were examined to determine if they are linked to behavioral intentions to perform activities advocated by the messages. Surprisingly weak linear relationships were uncovered between the two measures. However, the factor structure underlying the creative stimulus items is extremely consistent with that found by previous researchers. These results indicate that creative response measures are of limited usefulness for advertising copy-testing if behavior change is the objective.

Citation:

Gary L. Sullivan and P. J. O'Connor (1983) ,"Search For a Relationship Between Viewer Responses to the Creative Aspects of Televised Messages and Behavioral Intention", in NA - Advances in Consumer Research Volume 10, eds. Richard P. Bagozzi and Alice M. Tybout, Ann Abor, MI : Association for Consumer Research, Pages: 32-35.

Advances in Consumer Research Volume 10, 1983      Pages 32-35

SEARCH FOR A RELATIONSHIP BETWEEN VIEWER RESPONSES TO THE CREATIVE ASPECTS OF TELEVISED MESSAGES AND BEHAVIORAL INTENTION

Gary L. Sullivan, University of Cincinnati

P. J. O'Connor, Bernard Baruch College

ABSTRACT -

Subjects' responses to creative stimulus items for televised messages were examined to determine if they are linked to behavioral intentions to perform activities advocated by the messages. Surprisingly weak linear relationships were uncovered between the two measures. However, the factor structure underlying the creative stimulus items is extremely consistent with that found by previous researchers. These results indicate that creative response measures are of limited usefulness for advertising copy-testing if behavior change is the objective.

INTRODUCTION

A perennial problem for marketers involves the determination of the effectiveness of their media promotional efforts (Aaker and Myers 1975, Engel et al. 1979). Historically, it has been difficult to predict with confidence which campaigns will achieve success, and furthermore, what criteria should be used in determining communications success (Aaker and Myers 1976, Engel et al. 1979, Lavidge and Steiner 1961, and Lucas and Britt 1963). Long-standing debates have continued over the appropriateness of using behavioral criteria (e.g. sales variation) communication criteria (e.g. copy point recall, recognition, or wearout); or creative response criteria (e.g. attitudes or other evaluations of the advertisement itself) (Aaker and Myers 1975, Engel et al. 1919, Lavidge and Steiner 1961, Leavitt 1970, Leavitt 1975, Schlinger 1978, Schlinger 1979, Wells 1964, and Wells et al. 1971). Proponents of each of these approaches have developed their perspectives and techniques without consideration of the relationships which may exist between individual procedures. This study specifically examines the relationship between creative response criteria and behaviorally oriented criteria.

One of the more sophisticated and compelling approaches available to assess creative responses to advertisements is found in work by Leavitt (1970, 1975), Schlinger (1978, 1979), and Wells et al. (1971). In their research, each has developed a diagnostically useful measure of viewers' reactions to commercial messages. Operationally, subjects are presented with an advertisement in either completed or storyboard form and asked to rate the ad on a number of stimulus criteria. The procedures differ in two important respects. First, Leavitt and Wells employed adjectival stimuli for the ratings, whereas Schlinger favored complete sentences. Second, due to Schinger's use of sentence-type stimuli, instrument modification is required in some cases in order to be compatible with the content of the message.

These researchers had several different groups of subjects rate a large number of advertisements on a variety of criteria. The data acquired were factor analyzed (specifically, principle component analysis followed by varimax rotation) iteratively in order to reduce the stimuli to a reliable subset of items. Each researcher developed a final set of items which were found to have a high degree of consistency in fact.r structures across a number of investigations. Factor loadings of the stimulus items also exhibited a high level of interpretability.

The purpose of this study is to assess whether responses to creative stimulus items are related to behaviorally oriented criteria. Due to the more extensive literature available on adjectival creative response stimuli. an abridged version of the Leavitt (1975) adjective list was employed in this research. Use of adjectives also meant that a standard list could be used for a number of messages which differed in content. The adjective list, organized bs factors found by Leavitt, is shown in Table 1.

This study resulted as part of a larger project conducted for a state Department of Transportation. This project assessed the efficacy of a variety of highway traffic safety television public service announcements (PSA) in inducing compliance with the advocated behavior. Thus, a unique opportunity existed for investigating the relationship between two types of promotion effectiveness criteria, since the primary objective of the PSA is to induce behavioral change.

METHOD

Participants for the study were recruited from civic, religious, and/or fraternal organizations in a medium sized mid-western city. They were paid a nominal fee for their participation which they remitted to their sponsoring organization. In this way, subjects exhibiting a wide range of socio-economic and demographic backgrounds were obtained which should enhance external validity. The sample consisted of 155 individuals who were randomly assigned to one of four PSA message ( groups. For administrative purposes, the study was conducted on small groups of approximately 10 to 15 persons at a time.

At the beginning of the study subjects were told that the purpose of the investigation was to evaluate television advertising. Participants were not told who the sponsor of the research was until after the study was completed. Due to the ongoing data collection process. subjects were instructed not to discuss the nature of the study with others.

In a laboratory setting, subjects were shown a videotape of a popular network game show which contained the original commercials for several consumer products. The videotape also contained two exposures to one of the four PSA. These PSA were professionally edited into the tape and were separated by approximately twenty minutes of other program material. After viewing the television program, subjects received questionnaires to complete. Among other issues pertinent to the study sponsor's objectives, the questionnaire also elicited subjects' intentions to engage in specific highway / traffic safety behaviors as well as judgments about the creative aspects of the PSA using the abridged version of the Leavitt inventory which was described earlier. Scaled response formats were used for these questions to enhance analysis potential.

TABLE 1

ABRIDGED VERSION OF LEAVITT'S ADJECTIVE LIST

ANALYSIS AND DISCUSSION

Initial investigation of the factor structure of the descriptive adjective list used to rate the PSA was accompLished by a principal component analysis followed by varimax rotation. This is the procedure which was employed by previous researchers in their work. Since the concern focuses on creative responses to messages in general, rather than situation (or PSA) specific relationships, subjects from all four PSA message groups were aggregated for the factor analysis. Use of the Kaiser (1960) criterion of retaining factors with eigen-values greater than 1.0 produced a six factor solution accounting for 70% of the variance in the original ratings.

However, inspection of the resultant factor pattern reveals that only one of the adjectival stimuli loads on the sixth factor. Accordingly, a five factor solution explaining 66.5% of the original variance was judged to be more appropriate. The rotated factor pattern for this solution along with the associated eigenvalues is shown in Table 2. Clearly, this is a particularly clean factor structure in which all of the descriptive adjectives load heavily on one, and only one, factor. It is especially noteworthy that the solution obtained in this study corresponds closely with solutions obtained by a number of previous researchers cited above. Such a demonstration of stability in factor structures and factor loadings suggests strongly that these adJectives are reliable and valid for the purpose or evaluating informational mass media messages. External validity is enhanced by the fact that in this case the messages being rated are PSA rather than commercial messages as has been the case in prior investigations.

With respect to the source or the adjective inventory (Leavitt 1975), a high degree of consistency is evident. The eight adjectives which load heavily on the first factor are amusing, clever, exciting, lively, creative, imaginative, unique, and new. All of these adjectives, with the exception of new, were the only ones selected from the "amusing," "energetic." and "novel" subfactors of the stimulating factor (see Table 1). New was obtained from the familiar factor. The second factor contains the adjectives dull, sluggish, old, and repetitious. These were the only items selected from the "slow" and "worn out" subfactors or the simulating factor (see Table 1).

Convincing, helpful, meaningful to me, frank, worth remembering, believable, natural, and realistic, which loaded on factor three, were the only adjectives selected from the "personal relevance." "credible," and "realistic' subfactors of the relevant factor (see Table 1). The last adjective, informative, which loads on the third factor, comes from the "confusing" subfactor or the relevant factor (see Table 1). The adjectives in poor taste, silly, stupid, confusing, and unclear. which comprise factor four. were selected from the "irritating" and "confusing" subfactors of the relevant factor (see Table 1). The fifth factor consists of agreeable, attractive, and soothing; which come from the gratifying factor; and familiar which comes from the familiar factor (see Table 1).

As stated previously, the five factors uncovered (see Table 9) have high overlap with those round in other studies. Thus, they seem to tap similar content dimensions. The first and third factors, STIMULATING and RELEVANT are the most important accounting for 27.3% and 96.1% of the variance, respectively, MONOTONOUS and IRRITATING, the second and fourth factors, account for 16.7% and 17.5% of the variance, respectively, while factor five, LIKEABLE, accounts for 12.4% of the variance. These figures suggest that all of the factors are useful for analyzing mass media informational messages.

TABLE 2

FACTOR LOADINGS FOR PUBLIC SERVICE ANNOUNCEMENTS

The descriptive adjective inventory is useful for evaluating the content, structure, and execution of specific messages. However, no previous attempt has been made to explicitly assess the relationship between performance on the adjective inventory and conation. That is, haw performance on the adjective inventory for and particular message is related to intentions or actual behavior with respect to the subject of the message is unknown. As part of this research, questions were constructed to elicit respondents' intentions to conform to eleven specific behavioral activities advocated by the PSA. One of the PSA message groups responded to eight behavioral questions while the other three responded to one each. Subjects were asked to indicate on seven point Likert-type agree-disagree scales their intentions to follow the recommendations made in the PSA during the coming year.

Regression analyses were performed separately for each PSA message group. Eight of the regressions were performed using the behavioral intention measures responded to by the first message group as dependent variables. The remaining three regressions employed the single behavioral intention measure for their respective groups. The factor loadings reported earlier were used in conjunction with subjects' descriptive adjective ratings to obtain factor scores for each respondent. These five factor scores were then used as independent variables in the regression in order to ameliorate problems with multicollinearity in the original adjective ratings as well as degrees of freedom problems with small message group sizes. In all eleven cases, the models generated were non-significant in explaining variation in intentions among the respondents. Furthermore, R-square values were uniformly low for all the models. Inspection of the beta weights revealed that for all the models only the intercept term was significant. This indicates that the best estimate of an individual subject's intention score is the mean score across all subjects. Thus, respondents' evaluation of any particular message using the descriptive adjective inventory is not related, at least in a linear fashion, to their behavioral intention with respect to the action advocated in the message.

CONCLUSIONS

The results of this investigation should prove somewhat disconcerting to chose whose charge it is to develop effective advertising copy. Simple relationships apparently do not exist between commonly accepted creative response criteria and behavioral intentions. This study found no linear relationship between the five creative response factors and intentions. Thus, unless these creative issues are viewed as ends unto themselves, they should be of limited concern to the advertising strategist. A great deal more work on this subject is required. While the creative aspects of a message seem to be unrelated to the level of behavioral intention, their effect on changes in intentions, as well as on cognitive and attitudinal structures, needs to be investigated. Also, the interface between creative response criteria and communications criteria (e.g. copy point recall, recognition, wearout) deserves inquiry. Future studies should also be directed toward determining if a more complex relationship exists between creative response criteria and intentions. In sum, the relationships between creativity in televised messages and a multitude of eventual responses to such messages are highly complex.

REFERENCES

Aaker, D. A. and Myers, J. G. (1975), Advertising Management, Englewood Cliffs) NJ: Prentice-Hall.

Engel, J. F., Warshaw, M. R., and Kinnear, T. C. (1979), Promotional Strategy, 4th ed., Homewood, IL: R. D. Irwin.

Kaiser, R. J. (1960), "The Application of Electronic Computers to Factor Analysis," Educational and Psychological Measurements, 21 (Spring), 141-51.

Lavidge, R. J. and Steiner, G. A. (1961). "A Model for Predictive Measurements of Advertising Effectiveness," Journal or Marketing, 25 (October), 59-62.

Leavitt, C. (1970), "A Multidimensional See of Rating Scales for Television Commercials," Journal of Applied Psychology, 54, 947-99.

Leavitt, C. (1975), "Theory as a Bridge Between Description and Evaluation or Persuasion," in Advances in Consumer Research, Vol. 9, ed. M. J. Schlinger, Chicago: Association for Consumer Research, 607-13.

Lucas. D. B. and Britt, S. H. (1963), Measuring Advertising Effectiveness, New York: McGraw-Hill.

Sandage, C. H., Fryburger, V., and Rotzoll, K. (1979), Advertising Theory and Practice, 10th ed., Homewood, IL: R. D. Irwin.

Schlinger, M. J. (1978), "Attitudinal Reactions to Advertisements," in Attitude Research Under the Sun, ed. J. Eighmey, Tarpon Springs, FE: 9th Attitude Research Conference, 171-97.

Schlinger, M. J. (1979), "A Profile of Responses to Commercials," Journal of Advertising Research, 19 (April), 37-56.

Wells, W. D. (1964), "EQ, Son of EQ, and Reactions Profile," Journal of Marketing, 28, 45-52.

Wells, W. D., Leavitt, C., and McConville, M. (1971), "A Reaction Profile for TV Commercials," Journal of Advertising Research, 11 (December), 11-17.

----------------------------------------