Refining a Multidimensional Profile For Television Commercials: an Application of Targer Analysis

ABSTRACT - A survey was conducted to determine the dimensions which underlie consumer responses to television advertising. Fifteen groups of 46 subjects were each exposed to a different target commercial for a total of 690 ad exposures. One group of subjects rated ads using Leavitt's multidimensional profile, and a second group used Schlinger's Viewer Response Profile. The results indicated that the Leavitt profile performed somewhat better than the Viewer Response Profile, but there is still room for improvement in the former rating scales. Specifically, the fourth dimension of the Leavitt profile (Familiar) proves to be somewhat unstable and is a candidate for deletion.

Citation:

George M. Zinkhan and Scot Burton (1989) ,"Refining a Multidimensional Profile For Television Commercials: an Application of Targer Analysis", in NA - Advances in Consumer Research Volume 16, eds. Thomas K. Srull, Provo, UT : Association for Consumer Research, Pages: 711-718.

Advances in Consumer Research Volume 16, 1989      Pages 711-718

REFINING A MULTIDIMENSIONAL PROFILE FOR TELEVISION COMMERCIALS: AN APPLICATION OF TARGER ANALYSIS

George M. Zinkhan, University of Houston

Scot Burton, Louisiana State University

ABSTRACT -

A survey was conducted to determine the dimensions which underlie consumer responses to television advertising. Fifteen groups of 46 subjects were each exposed to a different target commercial for a total of 690 ad exposures. One group of subjects rated ads using Leavitt's multidimensional profile, and a second group used Schlinger's Viewer Response Profile. The results indicated that the Leavitt profile performed somewhat better than the Viewer Response Profile, but there is still room for improvement in the former rating scales. Specifically, the fourth dimension of the Leavitt profile (Familiar) proves to be somewhat unstable and is a candidate for deletion.

INTRODUCTION

Advertising researchers have shown considerable interest in identifying the dimensions which underlie viewers' reactions to television commercials (Leavitt 1970; Schlinger 1979; Wells, et al. 1971). Such profiles can be used to gauge viewers' immediate, initial reactions to ads and to select for airing commercials which viewers find enjoyable and entertaining. This is important since it has been shown that advertisers can gain a long-term competitive advantage by showing commercials that are enjoyable (Bartos 1981). Mitchell and Olson (1981) have found that immediate advertising response is an important mediator variable between ad exposure and attitude formation toward the advertised product. Through an attitude transfer mechanism, favorable feelings about the commercial can be translated into favorable feelings about the advertised product. This link between liking an ad and forming a good opinion of the advertised product is so strong that likable ads are becoming a major strategic tool in building "brand personality." That is, the advertiser that offends a viewer could face a serious negative bias in the marketplace.

Up until now, most of these commercial response profiles have been built up in an exploratory fashion, making use of a data reduction technique such as principal components analysis. Researchers have started out with as many as 525 items and gradually reduced this list to a smaller, more manageable set. When factor analysis is used in such an exploratory fashion, attempts to build or test measurement scales can amount to no more than tautological juggling of the input variables (Mulaik 1972). However, factor analysis need not be used only as an exploratory technique. Here, target analysis with Procrustes rotation -- a type of confirmatory factor analysis -- is used to test hypotheses about television response scales. An attempt is made to find out what basic dimensions underlie viewers' responses to commercials; and an attempt is made to determine if these responses can predict advertising effectiveness.

Leavitt's (1970) multidimensional profile, further refined at ACR (1975), is the most comprehensive method so far developed for capturing viewers' reactions to television commercials and is chosen for further study here. Since this profile was developed using exploratory analysis and since such analysis does not control for the impact of chance or sampling error, neither reliability nor validity has been demonstrated (Zinkhan and Fornell 1985). This is the first goal of the present study. Specifically, the 32-item profile originally proposed (Leavitt 1970) and then updated by Leavitt (1975) is investigated, and an attempt is made to determine if this multi-item profile can be accounted for by the same dimensions as originally hypothesized and to determine to what extent this profile useful as a predictor of advertising effectiveness.

Schlinger (1979) has proposed an alternative to the Leavitt profile which she terms the Viewer ; Response Profile (VRP). The VRP consists of 18 items and at least four stable dimensions (namely: Entertainment, Confusion, Relevant News, and Brand Reinforcement). The procedures for implementing the Leavitt and Schlinger profiles are very similar, but the items employed and the resulting dimensions extracted are quite different and sometimes conflicting. Thus, the second major goal of this study is to compare and contrast the two competing profiles, especially in terms of the consistency of their internal structure and [ in terms of their respective ability to predict consumer E choice behavior.

There have been a few previous attempts to examine the convergent and discriminant validity of I television commercial rating scales. For example, Lastovicka (1983) compared three Likert-type scales of [ three copy testing concepts (Relevance, Confusion, and Entertainment) with measures obtained from viewer 5 verbatim comments. He found evidence for the i measurement validity of two out of the three scales. Specifically, the relevance and entertainment structured questioning scales achieved satisfactory levels of reliability and convergent and discriminant validity. Lastovicka reported some problems with the Confusion dimension. Zinkhan and Fornell (1985) found that the Leavitt profile did not perform as well as one proposed by Wells (1964).

METHOD

The Profile

The items and dimensions which make up Leavitt's (updated in 1975) and Schlinger's (1979) profiles are shown in Tables 1 and 2. The four dimensions hypothesized to underlie the 32 Leavitt items include: Stimulating, Relevant, Gratifying, and Familiar. In the 1970 analysis performed by Leavitt, there were additional items used to operationalize the first two dimensions, but these are discarded in a later (1975) study, since they accounted for so little variance.

TABLE 1

TARGET MATRIX: LEAVITT PROFILE

A few of the dimensions in Schlinger's VRP have definite counterparts in the Leavitt scales. For example, both profiles include a dimension labeled Relevant (or Relevant News). However, the specific items used to operationalize this dimension are sometimes quite different when looking across the two profiles. Other dimensions appear unique to each respective profile. For example, there is no counterpart in Leavitt's profile for the Brand Reinforcement dimension hypothesized by Schlinger. Likewise, there is no obvious counterpart in the VRP for Leavitt's Familiar dimension.

The basic hypothesis underlying this study is that the 32 items shown in Table 1 and the 18 items from Table 2 will load as expected on the underlying dimensions. In addition, these two profiles will be compared and contrasted in terms of their ability to predict consumer choice behavior and other commonly used measures of advertising effectiveness.

Data

In order to test the two profiles, fifteen television commercials were selected. For each individual commercial, forty-six subjects were exposed to the commercial and proceeded to rate that ad using either the Schlinger or the Leavitt profile. Of the 46 subjects rating each commercial, 20 used the Leavitt profile (as shown in Table 1) and 26 used the Schlinger profile (as shown in Table 2). As in the original Leavitt (1970) study, unipolar scales were used to measure each item, and the items were randomly rotated so as to minimize any ordering effect. A 7-point scale was used for each item.

After completing the profile, respondents were given two additional measures of advertising effectiveness: attitude toward the brand (A(b)) and choice behavior (CB). A(b) was operationalized using three 7-point semantic differential scales (bad-good, unsatisfactory-satisfactory, and unfavorable-favorable). Choice behavior was operationalized by giving subjects the opportunity to select the advertised brand (or one of two competing brands) at the conclusion of the experiment. For example, if the target commercial was for a soft drink, subjects were offered a soft drink at the conclusion of the exercise and were given three options (one of which was the advertised brand).

TABLE 2

TARGET MATRIX: SCHLINGER PROFILE

Each subject was exposed to only one of the fifteen target ads and used either the Leavitt or the Schlinger profile. This procedure resulted in a sample size of 390 ad exposures for the Schlinger profile and 300 ad exposures for the Leavitt profile.

Analysis Procedure

Target analysis, a type of confirmatory factor analysis, is a promising technique for synthesizing and validating profiles that have been built up in an exploratory fashion. By using target analysis, it is possible to test hypotheses about the number of factors and to test hypotheses about the loading of each variable on each factor. The hypothesized factor pattern can be represented by a target matrix which specifies the direction and magnitude of each expected loading. Such a target matrix is represented in Table 1 for the Leavitt profile and in Table 2 for the Schlinger profile. Further details concerning target analysis, using an orthogonal Procrustes rotation, are given in the Appendix.

RESULTS

Convergent and Discriminant Validity

Tables 3 and 4 present the target analysis results. In general, the hypothesized factor structure is confirmed for the Leavitt profile. Reflecting the overall pattern and magnitude of loadings coefficients, the congruence (CC) is fairly satisfactory (.804). This indicates a strong similarity between the hypothesized target matrix and the obtained empirical results. While oblique Procrustes methods often show extremely high congruence, it should be kept in mind that the orthogonal rotation used here does not capitalize on "free" intra-set factor correlations. These correlations - are not allowed to "improve" the fit, and therefore are set to zero (Fornell, et al. 1981).

The first dimension, Stimulating, emerges as expected. However, two items, novel and unique, fail to load as strongly as hypothesized. The remaining 10 items show high positive loadings, as expected, with 8 of these loadings over .70.

The second and third dimensions also emerge as anticipated. For Relevant six of the eight items load above .60, and for Gratifying six of the eight items are above .75. In instances where a group of items does not load on a hypothesized factor, this may represent a failure in convergent validity. An example of this occurs in the second dimension where "dependable" and "worth remembering" load at only -.27 and .30 respectively. A similar breakdown occurs in the third dimension for "a good world" and "warm." Thus these four items are candidates for deletion.

Conversely, in instances where high or moderate loadings appear when zero correlations were expected, there is a problem with discriminant validity. An example of this appears in the fourth factor where "warm," "worth remembering" and "a good world" all load about .60. In addition to this discriminant validity breakdown in the fourth dimension, there is also a problem with convergent validity as none of the four familiar items loads very strongly. Thus, the entire fourth dimension is a candidate for deletion, and this represents the major failure in the analysis.

Beyond this breakdown in the fourth dimension, the results for the remaining items show rather strong evidence of convergent and discriminant validity. For the most part, the loadings which were hypothesized to be zero turned out to be insubstantial. Those loadings expected to be large turned out to be large, and the majority of loadings were signed in the hypothesized direction.

The results for the Schlinger profile are not quite so successful. As shown in Table 4, the coefficient of congruence is lower (CC = .710), and there are some problems with at least three of the four dimensions. For example, three out of seven items load below .50 on the Entertainment dimension, and none of the items on the Brand Reinforcement dimension work out as expected. In addition to these problems with convergent validity there are also some discriminant validity breakdowns (as revealed by the strong loadings of "I'm dissatisfied" on the Entertainment dimension and the strong loadings of "new product" and "not just selling" on the Brand Reinforcement dimension). Despite these inconsistencies, the Schlinger profile is somewhat successful in that the first three dimensions (Entertainment, Confusion, and Relevant News) emerge to some extent, with the Confusion dimension being the most successful of all (with all four loadings above .60). In general, however, the Schlinger solution does not seem quite as stable as the Leavitt solution. For both profiles, there are areas for improvement, which involve either the deletion of items or the deletion of entire profile dimensions.

Predictive Validity

Relationships between these two commercial profiles and other measures of advertising effectiveness were investigated. Specifically, it is expected that positive ratings on the profile dimensions should translate into positive attitudes toward the advertised brand and an increased likelihood of choice behavior (Shimp 1981; Bartos 1981). The Familiar dimension of the Leavitt profile is excluded from this analysis since it failed the previous tests of convergent and discriminant validity.

The Brand Reinforcement dimension of the Schlinger profile shared similar problems, but it is retained. Schlinger specifically noted the potential instability of Brand Reinforcement but argued for its retention since she found that Brand Reinforcement provided useful diagnostic information concerning other measures of advertising effectiveness. In both the Schlinger and Leavitt data sets, the three A(b) measures were collected into a single construct since the intercorrelations among these measures were high (average I = 0.81).

The original sample was split into two groups, so that 60% of the data were used for estimation and the remaining 40 percent were used as a validation sample. This procedure resulted in 180 Leavitt ad exposures and 234 Schlinger ad exposures being used for estimation and the remaining 120 Leavitt and 156 Schlinger ad exposures being used for prediction.

Estimation results for both profiles are shown in Table 5. Again, the Leavitt profile seems to slightly outperform the Schlinger profile. The three Leavitt dimensions account for almost 22% of the variance in brand attitude scores, while the four Schlinger dimensions account for over 20 percent of the variance in A(b). All the coefficients for the profile dimensions appear with a positive sign except for that associated with Confusion. This makes sense in that more confusing commercials are rated as less successful and result in lowered brand attitude scores. As revealed by the standardized beta weights, Entertainment is the most important Schlinger predictor and Stimulating is the most important Leavitt predictor.

In general, the profile dimensions are not as successful in accounting for choice behavior. Only one Schlinger dimension (Entertainment) proves to be a significant (p< .05) predictor of choice. However, when combined with brand attitude, the Leavitt dimensions explain over 32 percent and the Schlinger dimensions explain 24 percent of the variance in choice behavior scores.

In order to investigate the predictive validity of the proposed model, predictions were made to the holdout sample of 120 Leavitt and 156 Schlinger subjects. The models estimated in Table 5 were used to make these predictions. For the Leavitt profile, the correlation between predicted and actual values of A(b) and choice behavior are .36 (p< .01) and .34 (p< .01) respectively. For the Schlinger profiles, the values are .32 (p< .01) and .28 (p< .01).

TABLE 3

TARGET ANALYSIS RESULTS: LEAVITT PROFILE

TABLE 4

TARGET ANALYSIS RESULTS" SCHLINGER PROFILE

TABLE 5

RELATIONSHIP OF PROFILE DIMENSIONS TO OTHER MEASURES OF ADVERTISING EFFECTIVENESS

As is to be expected, there is some shrinkage in explained variance when moving from the original sample to the validation sample; but, in general, the prediction results are encouraging. There is some evidence of predictive validity.

DISCUSSION

These findings indicate that viewers' responses to advertising are multidimensional in nature. In most respects, the Leavitt commercial profile seems satisfactory in terms of convergent, discriminant, and predictive validity. However, the findings suggest some areas for improvement. Specifically, the Familiar dimension can be deleted due to instability. This result may arise partly from the fact that four items were suggested for operationalizing this factor. In future investigations, researchers may wish to add more items for tapping this dimension rather than deleting it altogether. Also, it seems as if the novel and unique items can be dropped from the first dimension, which proves to be the most important in terms of predicting other consumer responses to television commercials. The second and third dimensions are moderately important in terms of predictive power and also may be improved by the deletion of unnecessary items ("dependable" and "worth remembering" for the second dimension; "a good world" and "warm" for the third).

The Schlinger profile appears to require more adjustments than the Leavitt. In the former profile, only the Confusion dimension appears completely as expected. The Brand Reinforcement dimension is a candidate for deletion entirely, while Entertainment and Relevant News appear with as few as half their items loading as expected. The Schlinger items were selected originally because they represented verbatim playbacks supplied by actual viewers. Today, some of the items seem overly complicated (especially when compared to the single-word Leavitt items). It is some of those convoluted items (e.g., "the ad wasn't just selling the product -- it was entertaining me. I appreciated that") which provide some of the least satisfactory results. In fact, Schlinger did find some evidence that her profile dimensions were more or less stable, depending upon the particular sample of commercials selected. Perhaps the single-word approach used by Leavitt is a more preferable method of administration. Overall, the Leavitt profile appears more stable than Schlinger's VRP, and it does provide marginally better prediction of brand attitude and choice behavior.

Advertising researchers may want to take a closer look at print ad profiles, which also have been built up using exploratory analysis methods (see for example, Wells 1964). These profiles are widely used by advertising agencies, and yet their reliability and validity has not been demonstrated to any satisfactory extent (Zinkhan and Fornell 1985).

Target analysis, as employed here, proves to be a useful tool for investigating the immediate, short-term effects of advertising. Similar applications could be made in other areas where researchers have some a priori notions about the expected structure of a factor loadings matrix. Here the a priori notions were derived from previous empirical investigations; in other instances theory may be sufficiently developed to guide the formulation of a target matrix. In either case, target analysis is a simple yet powerful data reduction technique; such confirmatory analysis deserves to be considered whenever scientific knowledge begins to advance beyond the exploratory stage.

REFERENCES

Bartos, R. (1981), "Ads that Irritate May Erode Trust in Advertised Brands," Harvard Business Review, 138-140.

Cliff, N. (1966), "Orthogonal Rotation to Congruence," Psychometrika, 31, 33-42.

Fornell, C., John, G., Stern, L. W., & Triki, M. (1981), The Role Strain-Performance Relationship in Industrial Selling: A Process Analysis. Unpublished manuscript. Northwestern University, Chicago.

Green, B. F. (1952), 'The Orthogonal Approximation of the Oblique Structure in Factor Analysis," Psychometrika, 17, 429-440.

Lastovicka, John (1983), "Convergent and Discriminant Validity of Television Commercial Rating Scales," Journal of Advertising, 12 (No. 2), 14-23,52.

Leavitt, C. (1970), "A Multidimensional Set of Rating Scales for Television Commercials," Journal of Applied Psychology, 54, 427-429.

Leavitt, C. (1975), "Theory as a Bridge between Description and Evaluation of Persuasion," M.J. Schlinger (Ed.) (pp.607-13). In Advances in Consumer Research: Vol. 2, Chicago: Association for Consumer Research.

Mitchell, A. A. and Olson, J. C. (1981), "Are Product Attribute Beliefs the Only Mediator of Advertising Effects on Brand Attitude?" Journal of Marketing Research, 18, 318-332.

Mulaik, S. A. (1972), The Foundation of Factor Analysis. New York: McGraw-Hill.

Schlinger, M. J. (1979), "A Profile of Responses to Commercials," Journal of Advertising Research, 19, 37-46.

Schonemann, P. H. (1966), "A Generalized Solution of the Orthogonal Procrustes Problem," Psychometrika, 31, 1-10.

Shimp, T. (1981), "Attitude Toward the Advertisement as a Mediator of Consumer Brand Choice," Journal of Advertising, 10, 9-15.

Wells, W. D. (1964), "EQ, Son of EQ, and the Reaction Profile," Journal of Marketing, 28, 45-52.

Wells, W. D., Leavitt, C. and McConville, M. (1971), "A Reaction Profile for TV Commercials," Journal of Advertising Research, 11, 11-17.

Wrigley, C. and Newhaus, J. (1955), "The Matching of Two Sets of Factors," American Psychologist, 10, 418-419.

Zinkhan, G. and Fornell, C. (1985), "A Test of Two Consumer Response Scales in Advertising," Journal of Marketing Research. 22. 447-52.

APPENDIX: PROCRUSTES TARGET ROTATION

Following the work of Green (1952) and Cliff (1966) in rigid factor rotation, Schonemann's (1966) confirmatory Procrustes rotation is applied to the factor matrix which emerges. Thus, a matrix Z is formed from the target matrix T and the empirical (factor loadings) matrix L.

(1) Z = L ' T

We then extract the eigenvectors of ZZ, ZZ' and V and P, from the equations:

(2) ZZ' = P Y P

where Y is the diagonal matrix of eigenvalues. The transformation matrix is

(3) Q = P V'

The confirmatory Procrustes solution is given by

(4) T* = Q L

Since Q is chosen such that the matrix of errors (E = T - T*) is minimized in a least squares fashion, the solution is unique and may be tested for convergence (Fornell et al., 1981). To assess the similarity of T and T*, the coefficient of congruence (CC), as suggested by Wrigley and Newhaus (1955), is used. This measure is sensitive to pattern as well as magnitude differences in the two matrices. Values of CC will range from -1 to +1 and will be high when there is a high degree of fit between the observed loading matrix (T) and the expected loading matrix (T*).

----------------------------------------