Assessing Emotional Reactions to Tv Ads: a Replication and Extension With a Brief Adjective Checklist

Haim Mano, University of Missouri - St. Louis
ABSTRACT - This study examines the replicability of a brief checklist (yes/no) version of a 11-point scales' questionnaire for assessing emotional reactions to TV advertisements. By applying two different methodological paradigms, classification and dimensionality, the analyses revealed that, as with the longer form questionnaire, the structure of feelings experienced during exposure to TV ads can be described by affect's two primary dimensions, Pleasantness and Arousal. Based on this structure, a simple method designed to assess the emotional intensity of TV ads is offered. The method is tested by comparing emotional reactions to first-time and previously seen ads. The results suggest that, overall, previously-seen ads were liked more than first-time seen ads.
[ to cite ]:
Haim Mano (1996) ,"Assessing Emotional Reactions to Tv Ads: a Replication and Extension With a Brief Adjective Checklist", in NA - Advances in Consumer Research Volume 23, eds. Kim P. Corfman and John G. Lynch Jr., Provo, UT : Association for Consumer Research, Pages: 63-69.

Advances in Consumer Research Volume 23, 1996      Pages 63-69

ASSESSING EMOTIONAL REACTIONS TO TV ADS: A REPLICATION AND EXTENSION WITH A BRIEF ADJECTIVE CHECKLIST

Haim Mano, University of Missouri - St. Louis

ABSTRACT -

This study examines the replicability of a brief checklist (yes/no) version of a 11-point scales' questionnaire for assessing emotional reactions to TV advertisements. By applying two different methodological paradigms, classification and dimensionality, the analyses revealed that, as with the longer form questionnaire, the structure of feelings experienced during exposure to TV ads can be described by affect's two primary dimensions, Pleasantness and Arousal. Based on this structure, a simple method designed to assess the emotional intensity of TV ads is offered. The method is tested by comparing emotional reactions to first-time and previously seen ads. The results suggest that, overall, previously-seen ads were liked more than first-time seen ads.

Television advertisements are filled with emotional messages that can evoke a broad spectrum of emotional experiences. Many TV ads contain affective stimuli as background features (e.g., pleasant music, "touching" stories) and, in many, affect is the ad's main message (e.g., fun, threat). Because of their ease of replicability and the various degrees of intensity of emotional responses they can evoke, TV ads also have practical advantages for examining the structure and intensity of affect. Indeed, since TV ads are unanticipated external stimulations intentionally targeted at altering the viewer's emotional state, watching them can allow mood induction(s) by exposing subjectsCover a relatively short period of timeCto a number of diverse emotional stimuli of controlled nature and varied intensity.

STUDY OBJECTIVES

The present study has four objectives. First, to propose and examine the validity and suitability of a simple checklist (yes/no) questionnaire for assessing the emotional impact of TV ads and other mood evoking stimuli. The examined checklist questionnaire is based on an instrument developed by Mano (1991) that used 11-point items. A short adjective checklist offers two advantages that facilitate the judgmental processes required for reporting one's emotional state: (i) a relatively small number of adjectives and (ii) the fast and easy response mode (simple check) without the need to quantify the extent to which a particular emotion was felt during exposure to the stimulus. In contrast to longer and multi-point items, these may be critical considerations particularly since TV ads usually last no more than 30 seconds and many times invoke only limited-intensity and short-lived emotional reactions. (For another approach that uses a simple graphic tool for assessing emotional reactions see the Affect Grid; Russell Weiss and Mendelsohn 1989).

The second goal, and part of the questionnaire's validity examination, is to further assess the evidence on the convergence and complementarity of the two paradigms of the structure of affect, dimensionality and classification. Replicating and extending Mano (1991), the paradigms will be contrasted by applying three alternative structure-probing methods: (1) multidimensional scaling of correlations (MDS), intended to reveal the items' primary underlying spatial dimensions, (2) factor analysis, aimed at classifying and dimensionally describing the items, and (3) cluster analysis, aimed at dividing items into clusters of similar groups. The three-method approach allows to contrast the paradigms and examine various aspects of affect's structure emphasized by each. A central question of psychometric concern of this study is the structural invariance of the factorial and other dimensional solutions under different response formats. In particular, there are two major differences between a multipoint (like the one used by Mano 1991) and a checklist (yes/no) format. First, responding to whether a particular feeling was experienced is not conceptually identical to rating the strength of that feeling. And, second, restriction of response range in the yes/no format strongly attenuates inter-item correlations which in turn could have had a profound effect on the results of correlation-based methods used here (MDS and factor analysis).

The third objective is to apply the proposed instrument for assessing the emotional intensity of different TV ads. Finally, the fourth objective is to examine whether and how prior familiarity with an ad can influence emotional responses to it. Despite theoretical and empirical implications for advertising, past research has not addressed the relationships between ad-familiarity and affect. Zajonc (1968) suggested that repeated exposure leads to liking. The question could therefore be raised as to what aspects of the emotional spectrum are influenced by repeated experience. In particular, since liking involves positive hedonic tone, it is expected that previously seen ads would lead to an increase of the Pleasantness dimension.

DIMENSIONALITY AND CLASSIFICATION OF EMOTIONS

Research on the relationships among emotions usually endorses one of two paradigms, dimensionality or classification. The dimensional view describes emotions in terms of a minimal number of basic dimensions. A substantial body of research suggested Pleasantness and Arousal (Russell 1980) or the 45-degree rotation of Positive and Negative Affectivity (Watson and Tellegen 1985) as affect's primary dimensions (Figure 1).

To date, a number of studies have found that these two dimensions and their combinations underly emotional reactions to TV ads (e.g., Batra and Holbrook 1990, Edell and Burke 1987, Holbrook and Batra 1987, Mano 1991, Olney Holbrook and Batra 1991) and product consumptions (Mano and Oliver 1993).

The parsimony and generality of few dimensions, however, is challenged by those advocating for comprehensive classifications of emotions (e.g., Clore Ortony and Foss 1987). The need to cover a broad spectrum of emotions stems from the multitude of emotional stimuli found in TV ads. As a result, a unique feature of past research has been the large number of items used to assess the structure and intensity of ad-evoked emotional reactions (e.g., Batra and Holbrook 1990, Batra and Ray 1986, Holbrook and Batra 1987).

In attempting to compare and contrast the classification and dimensional paradigms, however, it is important not to view them as antithetical but as complementary (Mano 1991; Russell 1980). Some of the contrasts between the paradigms could be reconciled if differences between their respective methodologies were taken into account. Procedurally, dimensionality relies on the primary dimensions revealed in MDS analyses of judgments of similarity or on the principal components found in factor analysis of emotional items. Classification, on the other hand, examines the results and interpretability of full factorial solutions (i.e., all factors with eigenvalue>1) or cluster-analytic techniques. In order to contrast the paradigms, Mano (1991) applied dimensional and cluster analytic methods on emotions elicited during lecture attendance and by exposure to TV ads. The results suggested a convergent and complementary understanding of the paradigms. Classification solutions identified Good Mood, Bad Mood, Arousal, Quietness, Elation, Distress, Calmness, and Boredom as interpretable groups of emotions while the two-dimensional solutions parsimoniously described the relationships among these groups as depicted in Figure 1.

FIGURE 1

THE AFFECT CIRCUMPLEX

METHOD

Subjects-Judges. Eighteen subjects (7 females, 11 males) were recruited by announcements posted in a midwestern university campus. They were run individually and paid $7.50 for participating in a session that lasted about 50 minutes.

Advertisements. Forty one national and local ads (the same ones used in Mano, 1991) were used as the affect inducing stimuli. Some of the ads lacked strong emotional content and were informational. Most ads lasted 30 seconds. To enhance generalizability, the ads were randomly assigned to two series of 21 (Tape 1) and 20 (Tape 2) presented to two different groups of judges.

Instrument. After seeing each ad, subjects were presented with a list of the emotion describing adjectives used by Mano (1991) and asked to indicate (by placing an X) whether they experienced that particular emotion while watching the ad. Three or four items represented each region in the circumplex (Arousal: Astonished, Surprised, Aroused; Elation: Elated, Active, Excited; Pleasantness: Pleased, Satisfied, Happy, In Good Mood; Calmness: Calm, At rest, Relaxed; Quietness: Quiet, Still, Quiescent; Boredom: Sleepy, Sluggish, Drowsy; Unhappiness: Unhappy, Sad, Blue, In Bad Mood; Distress: Anxious, Fearful, Nervous).

Procedure. Nine subjects (Group 1) viewed and responded to Tape 1 and the other 9 (Group 2) to Tape 2. After viewing each ad, the subject paused the VCR and responded. The tapes were professionally edited to include a short 10-seconds blank (blue) screen between ads. The data were gathered on a booklet in which each page corresponded to an ad with the Product/Ad name at the top. The twenty six adjectives were presented in random order (same for all ads) and subjects were requested to place an X near any of the adjectives that best described their feelings after watching the ad. Subjects were then prompted to add in writing any additional emotions felt while watching that ad not mentioned in the list (places were provided for four additional traits).

Ad Familiarity. At the bottom of each page, subjects indicated whether they had seen the ad in the past.

RESULTS

The basic units of analysis were the responses to the 41 ads made by the judges. Responses were coded as 1 (if the trait was marked) or 0. The data consisted of 189 sets of responses for Group 1 (9 subjects X 21 ads) and 180 (9 X 20) for Group 2. Each set contained the 26 original and any other items noted by the subject. Content analysis of the additional items revealed that Bored/Boring was mentioned in 4.6% of the responses (across all ads and subjects) and was added as the 27th item. Of the 27 items, subjects marked, on average, 4.87 items per ad.

For purposes of structural analysis, combining assessments for the same ads and across the same subjects introduces some degree of dependence. To detect this dependence, the robustness of the MDS, factor, and cluster analyses between groups was first assessed (the dichotomous response format does not allow for within-subject standardization of responses aimed at eliminating between-subjects variability). As the results showed, there was considerable factorial invariance and a particularly strong cluster and dimensional-scaling invariance between the two Groups thus allowing for across-group aggregations.

MDS. For Group 1, the stress coefficients of the MDS of the item correlation matrix, for dimensions 1 to 4, were .49, .18, .11, and .08 (explaining 43%, 84%, 90% and 94% of item variance); for Group 2, .52, .25, 0.17 and .12 (33%, 70%, 77% and 83%). Based on stress reduction and interpretability, both two-dimensional solutions were considered appropriate. The emerged dimensions in both maps were Pleasantness and Arousal. The two maps were very similar: the Pleasantness coordinates correlated .92 and the Arousal coordinates .94. The canonical coefficient between the two sets of coordinates was .97. Given this high resemblance, the combined data sets were subjected to MDS. The stress coefficients for the first four dimensions were 0.50, .19, .13, and .09 (40%, 83%, 89%, and 93% of the variance). The two-dimensional solution (Figure 2) deemed appropriate in terms of stress reduction and interpretability, with Pleasantness and Arousal emerging as the underlying dimensions. One of this solution's most salient features is its high visual similarity with the theoretical model in Figure 1.

FIGURE 2

TWO-DIMENSIONAL SCALING SOLUTION OF THE 27 ADJECTIVES

Factor Analysis. Separate factor analyses for Groups 1 and 2 revealed moderately high factorial invariance across groups. In the combined data set, seven factors with eigens>1 were revealed, accounting for 56% of the variance; the respective eigenvalues were 2.82, 2.58, 2.53, 2.18, 1.84, 1.73, 1.34. Items and loadings contributing to each factor were: Sleepy .80, Bored .76, Sluggish 0.72, Drowsy .59 (interpreted as Boredom); Excited .82, Active .72, Elated .64, Aroused .58 (Elation); Nervous .77, Anxious .72, Fearful .62 (Distress); Quiet .69, Calm .67, Still .64, At rest .58 (Calm-Quiet); Blue .76, Sad .73 (Sad); Astonished 0.76, Surprised .72 (Surprised); and Quiescent .74. The following items did not load above .5 on any of the factors: Relaxed, Satisfied, Pleased, Happy, Unhappy, In good mood, and In bad mood.

A number of observations regarding the theoretical appeal of the seven-factor solution are pertinent. First, all factors were monopolar. Second, the first two pairs of factors (Bored-Elated, Distressed-Calm) contained conceptually opposed items which, in the MDS solution, were diametrically opposed. Third, all items loading above .5 in a factor appear in spatial contiguity in the two-dimensional scaling solution. Fourth, items conceptually related to Pleasantness-Unpleasantness were not included in any factor. Finally, the seventh factor (Quiescent) contained only one item with>.5 loading, indicative that this may not be a meaningful factor. Taken together these results suggest that a lower order factor solution may be theoretically more justifiable. Eigenvalue reduction in the seven-factor solution, however, was rather smooth and therefore it is not clear how to reduce the number of factors.

Given the predominance of two dimensions in the MDS solution, it is theoretically appealing to examine the congruence between the two primary factors and the two-dimensional solution. To that goal, the varimax-rotation constrained-to-two-factors analysis of the data was conducted. The resulting factorial configuration was very similar to the two-dimensional solution presented in Figure 2 (the canonical correlation between the MDS and factor-analytic sets of coordinates was .996).

Taken together, the results indicate that the two-dimensional and the two-factor solutions were very similar. Also, the positions of the 27 adjectives in both solutions corresponded with the configuration suggested by the theoretical model presented in Figure 1.

At this point it is also noteworthy that the canonical correlations between the coordinates of the MDS and two-factor solutions in this study and the corresponding MDS and two factor solutions in Mano (1991) were extremely high ranging from .976 to .985.

Cluster Analysis. The category-sort task is commonly used for revealing typologies of semantic structures. In it, subjects sort terms according to their perceived similarity into non-overlapping groups and the across-subjects co-occurrence matrix is submitted to a cluster analytic method. The adjective checklist used here allows to treat responses to a particular ad as sorted into two groups (checked and non-checked). By counting (across ads and subjects) the number of times each pair of items co-occurred, a matrix of item co-occurrences can be derived whereby a higher value of a matrix entry reflects a higher degree of item co-occurrence. Note that (a) co-occurrence matrices are different from the across ads and subjects inter-correlation matrices used in MDS and factor analyses, and (b) they do not merely yield the structure of semantic meanings but, rather, the structure of the affective experiences associated with the affective stimuli.

FIGURE 3

HIERARCHICAL CLUSTERING OF THE FEELINGS EXPERIENCED WATCHING THE 41 TV ADS

The across-ads-and-subjects co-occurrence matrices of Groups 1 and 2 were first separately submitted to Johnson's (1967) hierarchical cluster analysis. The two hierarchical structures were very similar and, therefore, the two populations were pooled and their matrix re-submitted to the analysis. The results of the combined matrix are presented in Figure 3.

Figure 3 suggests that all items contained in the revealed clusters were adjacent on the two-dimensional-factorial circumplex. Moreover, the various same-branch clusters were also on contiguous regions of the circumplex. Thus, the hierarchical cluster analysis revealed that the examined emotions can also be aptly described in terms of the circumplex.

Synthesis. Taken together, these results suggest the suitability of the brief checklist format for assessing the emotional impact of TV ads or other emotional stimuli. In particular, the findings allow the aggregation of the items into eight scales: Arousal (Astonished, Surprised, Aroused); Elation (Elated, Active, Excited); Pleasantness (Pleased, Satisfied, Happy); Calmness (Calm, At rest, Relaxed); Quietness (Quiet, Still, Quiescent); Boredom (Sleepy, Sluggish, Drowsy); Unpleasantness (Unhappy, Sad, Blue); and Distress (Anxious, Fearful, Nervous). The two-dimensional scaling solution of the correlations among the eight scales is presented in Figure 4 (stress=.039); as seen, the solution highly resembles the circular configuration of Figure 1. Thus, aggregation of the items into the eight scales further reinforces the generalizability of the circumplexial model of affect.

FIGURE 4

TWO-DIMENSIONAL SCALING SOLUTION OF THE OCTANT SCALES

APPLICATIONS AND IMPLICATIONS

Assessing Emotional Impact

Item aggregation into the eight-scale profile can be applied to assess different aspects of TV ads (or other emotion eliciting stimuli). These include across-ads or within-ad comparison and assessment of salient emotional features. To this goal, consider in Table 1 the emotional tone of some of the ads and all the ads used in the study. Such descriptions on the scales can form the basis for classifying the ads by any relevant a-priori criteria (e.g., elating, calm, sad and distressing, etc.). Furthermore, correlations between stimuli based on the eight scales can provide a measure of emotional similarity of the stimuli; subsequently, intercorrelations of ads can be used as the basis for ad-hoc segmentation or other forms of cluster analysis.

Based on the responses given to an ad across the eight scales (i.e., total number of items checked per ad), we can also assess how ads differ in their total emotional impact, an index that can capture the overall emotional intensity of an ad. For example, the lowest average number of responses for an ad was 3.33 (s.d.=1.73), while the highest was 7.00 (s.d.=2.45).

Table 1 also shows the total number of responses on the scales for all the ads used in this study. Although the 41 ads may not consist a representative sample, they generally reflect many of the typical themes found in TV ads. The most salient feature of the aggregation of the 41 ads is their strong emphasis on Pleasantness. It appears that advertisers prefer to create and show pleasant ads more often than any other type of ads.

Familiarity Effects. Based on subjects' report that they had seen the ad before, familiar and first-time seen ads were compared on each of the 27 items (Table 1). Of the 369 observations, 130 were previously seen ads. Across subjects and ads, significant familiarity differences were revealed for the four following items: Satisfied, noted for 48% of the 130 previously seen ads versus 34% for the 239 unfamiliar ads (p=.008); Happy (55% vs. 39%, p=.005); In Good Mood (60 % vs. 45%, p=.006); Bored (1% vs. 7%, p=.009). The first three items belong to the domain of Pleasantness, suggesting that prior familiarity with an ad can enhance its positive hedonic tone and emotional appeal. Familiar ads were also judged less boring than unfamiliar ads. This result stands in some contrast with the common notion that boredom is more likely to be associated with familiar stimuli and that novel stimuli would be more interesting. It seems that, in general, prior familiarity with an ad renders it more pleasing and less boring than a first-time seen ad. These findings suggest that prior familiarity with an ad enhanced its likeability.

Two issues are noteworthy about this relationship. First, since subjects stated ad recognition and not familiarity (i.e., the extent of their prior exposure), these results could be strictly interpreted as suggesting a link between likeability and recognition. Nonetheless, since most TV ads are shown and viewed many times, the link may actually be between repeated exposure and likeability. Clearly, the links between emotional reactions and the extent of prior exposure to an ad, likeability, and subsequent ad-wearout require further research.

The second issue concerns the predominance of pleasantness in the ads and the earlier suggestion that advertisers prefer to show pleasant ads. This statement should now be qualified by the positive relationship between prior exposure and likeability: if previously seen ads are liked more, then ad likeability could also be stemming from ad familiarity. To examine this issue closer, the scores on the eight scales were compared for the previously seen and the first-time seen ads (Table 1). These comparisons revealed significant differences only for the Pleasantness scale: previously seen ads were perceived as more pleasant than first-time-seen ads; (1.59 out of the possible 3 vs. 1.23; t(367)=2.88, p<.005). Thus, at least some of the likeability of previously seen ads should be attributed to subjects' familiarity with them. Nonetheless, it should also be pointed out that for the first time seen ads too, the scale of Pleasantness had the highest score (1.23), thus reinforcing the notion that, overall, advertisers prefer to show pleasant ads.

LIMITATIONS

Some of the limitations of the present research should be noted. First, despite the convergence of the two-factorial solutions with previous research, the disadvantages of dichotomous data should be acknowledged. In particular, attenuation of correlations induced by a yes/no (as opposed to a multi-chotomous) format precludes effective estimation of scale reliability. Also, attenuated correlations raise the issue whether factor analyses should be applied to such data. For example, the gradual descent in eigenvalues in the full (seven) factor solution should be mostly attributed to correlation attenuation. Or, consider, that while Pleasantness typically emerges as the primary factor in full factorial solutions of emotional adjectives, the present full factor solution did not include Pleasantness as a distinct factor; or, that in the full factorial solution, the item Quiescent emerged as a unique factor. Taken together, these findings suggest that caution should be applied when attenuated correlations are subjected to factorial analyses.

TABLE 1

RESPONSES ON THE OCTANT-SCALES

A second issue concerns whether the intensity of some emotional experiences are adequately captured by the yes/no response. For example, some emotions may have been checked but experienced at a relatively weak level by one subject, whereas, another subject, may have had a strong emotion associated with only one of the adjectives. Perhaps, in addition to intensity, the suggested scales also measure an ad's emotional diversity.

A third, and related, limitation is that the validity of any solution ultimately depends on the validity of the items; (for example, "tired", "sleepy" and "quiescent" may not be considered appropriate exemplars of affective adjectives). Thus, a specific sample of items and their semantic similarity could "drive" the two dimensional configuration (Clore Ortony and Foss 1987). Nonetheless, it should be noted that even when using much larger batteries of traits, Pleasantness and Arousal (or Positive and Negative Affectivity) still emerge as the primary dimensions of emotions (e.g., Holbrook & Batra 1987, Watson and Tellegen 1985).

Fourth, given the self-report and correlational nature of the present data, it is not clear whether increased liking for previously seen ads is driven only by familiarity or by some simple inferential process. Future research should experimentally control ad familiarity and incorporate samples of ads representative of TV programs. For example, the stimuli could consist of all prime time ads shown by a TV station during one evening combined with new (unfamiliar) ads and/or ads from other parts of the country.

Finally, the possible differences between emotional reactions invoked while versus after watching an ad need to be highlighted. Even though subjects were requested to report their feelings while watching the ad, the reports were made after seeing the ad. This raises the issue of whether the items captured the summative mood effects ("after") or, rather, reactions to some of the ad's emotional elements ("while"). It is possible that summative mood effects may contain fewer checked items than emotions experienced while watching the ads, thus resulting in smaller correlations (or fewer co-occurrences) between items under the "after" scenario.

CONCLUSION

The present study suggests that the proposed checklist questionnaire is suitable for assessing the structure and intensity of emotions elicited by TV ads. Two central findings of this study are (1) the high degree of structural convergence revealed across methods and (2) the strong agreement that the circumplex of affect is the basic structure of the examined emotional experiences. As seen, factorial solutions and cluster analyses identified interpretable groups of emotions while, at the same time, the two-dimensional solutions parsimoniously described the relationships among the items in these groups as well as the interrelationships among the groups. Finally, an application of the proposed tool suggested that previously seen ads are, overall, liked more than first-time seen ads.

REFERENCES

Batra, Rajeev and Morris B. Holbrook (1990), "Developing a typology of affective responses to advertising: A test of validity and reliability," Psychology and Marketing, 47 (1), 11-25.

Batra, Rajeev and Michael L. Ray (1986), "Affective response mediating acceptance of advertising," Journal of Consumer Research, 13, 234-249.

Clore, G.L., Ortony, A., and Foss, M.A. (1987), "The psychological foundations of the affective lexicon," Journal of Personality and Social Psychology, 14, 421-433.

Edell, Julie A. and Marian C. Burke (1987), "The power of feelings in understanding advertising effects," Journal of Consumer Research, 14, 421-433.

Holbrook, Morris B. and Rajeev Batra (1987), "Assessing the role of emotions as mediators of consumer responses to advertising," Journal of Consumer Research, 14, 404-420.

Johnson, S.C. (1967), "Hierarchical clustering schemes," Psychometrica, 42, 241-254.

Mano, Haim (1991), "The structure and intensity of emotional experiences: Method and context convergence," Multivariate Behavioral Research, 26 (3), 389-411.

Mano, Haim and Richard L. Oliver (1993), "Assessing the Dimensionality and Structure of the Consumption Experience: Evaluation, Feeling, and Sartisfaction," Journal of Consumer Research, 20 (December), 451-466.

Olney, Thomas J., Morris B. Holbrook, and Rajeev Batra (1991), "Consumer Responses to Advertising: The Effects of Ad Content, Emotions, and Attitude toward the Ad on Viewing Time," Journal of Consumer Research, 17 (March), 440-453.

Russell, James A. (1980), "A circumplex model of affect," Journal of Personality and Social Psychology, 36, 1152-1168.

Russell, James A., Anna Weiss and Gerald A. Mendelsohn (1989), "Affect grid: A single-item scale of pleasure and arousal," Journal of Personality and Social Psychology, 57, (3) 493-502.

Watson, David and Auke Tellegen (1985), "Toward a consensual structure of mood," Psychological Bulletin, 98, 219-235.

Zajonc, Robert B. (1968), "Attitudinal effects of mere exposure," Journal of Personality and Social Psychology Monograph Supplement, 9 (2), 1-27.

----------------------------------------