A Multitrait-Multimethod Analysis of the Validity of Cognitive Response Assessment Procedures

ABSTRACT - Cognitive response research is currently enjoying its "hey-day," in product life-cycle terms. The present paper reports the results of two conceptually similar studies which addressed the validity of the independent Judges method and the subJect self-rating method for assessing cognitive responses. Overall, both methods were found to be valid. However, it was also round that percentage score measures, relative to gross and net frequency measures, led to less discriminant validity.

Citation:

H. Bruce Lammers (1985) ,"A Multitrait-Multimethod Analysis of the Validity of Cognitive Response Assessment Procedures", in NA - Advances in Consumer Research Volume 12, eds. Elizabeth C. Hirschman and Moris B. Holbrook, Provo, UT : Association for Consumer Research, Pages: 164-172.

Advances in Consumer Research Volume 12, 1985      Pages 164-172

A MULTITRAIT-MULTIMETHOD ANALYSIS OF THE VALIDITY OF COGNITIVE RESPONSE ASSESSMENT PROCEDURES

H. Bruce Lammers, California State University, Northridge

[Note: This research was supported, in part, by a research grant from the CSUN School of Business Administration and Economics Research and Grants Committee. The author thanks Harold H. KassarJian for his comments and assistance on this project. Address correspondence to H. Bruce Lammers, Professor of Marketing, Department of Marketing, California State University, Northridge, Northridge CA 91330.]

ABSTRACT -

Cognitive response research is currently enjoying its "hey-day," in product life-cycle terms. The present paper reports the results of two conceptually similar studies which addressed the validity of the independent Judges method and the subJect self-rating method for assessing cognitive responses. Overall, both methods were found to be valid. However, it was also round that percentage score measures, relative to gross and net frequency measures, led to less discriminant validity.

INTRODUCTION

Gauging the quantity and quality of cognitive response output has become a useful and celebrated method of monitoring cognitive processing both in social psychology and consumer behavior research (Kassarjian 1982; Petty, Ostrom, & Brock 1981; Wright 1980). Moreover, interest in cognitive response assessment has transcended academic curiosity and has become a successfully applied tool in industry ("Ford has a Better Idea...," 1981). The purpose of the present paper is not to review the cognitive response literature, for excellent comprehensive reviews of the cognitive response research can already be found in social psychology (Petty, Ostrom, & Brock 1981) and in consumer behavior (Wright 1980). Rather, the present paper represents a formal examination of the construct validity of two frequently-used methods of cognitive response assessmentCthe independent judges method and the self-rating method. In so doing, the present paper extends Swasy's (1980) research on the construct validity of to mon cognitive response indices.

Methods of Cognitive Response Assessment: Judges vs. Self

With the independent Judges method, the ratings of the cognitive responses are typically done by several outside "Judges" or raters. An advantage of the independent Judges method is that the Judges can often be efficiently and effectively trained on a given task. With the self-rating method the respondents rate their own cognitive responses. The usual purpose of both the independent judges method and the self-rating method is to determine the evaluative direction (pro vs. con vs. neutral) and/or intensity of each cognitive response recorded by the respondent. (For a fine review of other variations in cognitive response measurement techniques see Cacioppo, Harkins, & Petty 1981; Cacioppo & Petty 1981; and, Miller & Baron 1973).

Neither method appears to be significantly more popular in usage than the other. For example, an inexhaustive list of those studies which used the independent Judges method includes Belch (1981, 1982), Belch and Belch (1984), Brock (1967), Cacioppo (1979), Cacioppo & Petty (1979a, 1980), Chaiken (1980), Chaiken and Eagly (1976), Cook (1969), Eagly (1974), Eagly and Warren (1976), Edell and Mitchell (1978), Edell and Staelin (1983), Fitzpatrick & Eagly (1981), Harkins and Petty (1981b), Heesacker and Petty (1983), Insko, Turnbull, and Yandell (1974), Keating and Brock (1974), Kelman (1953), Lammers (1982, 1983), Lammers and Becker (1975), Lammers, Seymour, and Wilkinson-Lammers (1981), Marks and Olson (1981), McCullough and Ostrom (1974), Olson, Toy, and Dover (1978, 1982), Osterhouse and Brock (1970), Percy and Lautman (1981), Petty (1977), Petty and Cacioppo (1981), Petty, Cacioppo, and Heesacker (1981), Petty, Cacioppo, and Schumann (1983), Petty, Wells, and Brock (1976), Roberts and Maccoby (1973), Romer (1979a), Shavitt and Brock (1984), Smith and Hunt (1978), Swasy (1980), Toy (1982), Wilson and Muderrisogulu (1980), Wright (1973, 1974, 1975), Wright and Rip (1980, 1981). Examples of cognitive response research which used the self-rating method include Cacioppo and Petty (1979b), Cacioppo, Petty, and Quintanar (1982), Calder, Insko, and Yandell (1974), Carment and Foster (1969), Cialdini, Levy, Herman, Kozlowski, and Petty (1976), Greenwald (1968), Harkins and Petty (1981a), Harmon and Coney (1982), Insko, Lind, and LaTour (1976), Lammers (1979), Lammers, Leibowitz, Seymour, and Hennessy (1983), Love and Greenwald (1978), Madden and Debevec (1983), Petty and Cacioppo (1979a, 1979b), Romer (1979a, 1979b ), Shavitt and Brock (1984), Sternthal, Dholakia, and Leavitt (1978).

Despite the rather obvious popularity of the two methods, little attention has been directed at formally assessing the convergent and discriminant validities of the two methods. On the other hand, the issue of validity has not been entirely ignored. At least one cognitive response study (Petty, Wells, & Brock 1976) did use both methods and found that significant convergent validity existed between the two. It is intriguing to discover that the Petty, Wells, and Brock (1976) study is often cited as the sole empirical Justification for using either method. However, this is a heavy burden to place on a single study, for there are sound, theoretical reasons for hypothesizing the existence of significant discrepancies between the ratings of outside Judges and from the subjects themselves (Wright 1980). Perhaps the forerunner among the alternative theories is attribution theory, which thrives on actor-observer (subJect-Judge) differences in explanations of behavior (Harvey, Ickes, & Kidd 1976; Herzberger & Clore 1979; Mizerski, Golden, & Kernan 1979). In a number of situations, observers are more likely than actors to view actors' behavior as being internally caused. Conversely, actors are more often likely than observers to view their own behavior as being more influenced by external factors. This pervasive tendency for observers to "overestimate the importance of personal or dispositional factors relative to environmental influences" (Ross 1977, p. 184 ) is called the fundamental attribution error (Jones 1979).

In the present context, the fundamental attribution error hypothesis suggests that independent judges (observers) are likely to emphasize internal causes, e.g., internalized beliefs and affects, for the cognitive responses expressed by others. In so doing, the Judges are likely to rate the cognitive responses in a more polarized fashion than the subjects themselves. That is, the independent Judges may have a tendency to overestimate the negative and positive tones of cognitive responses. This could result in more cognitive responses being rated as pro- or counterarguments than as neutral or irrelevant, relative to the ratings assigned by the subjects themselves. The potential for conflicting data from the two methods calls for an examination of their convergent, discriminant, and criterion-related validities. The present Paper reports the results of two conceptually similar studies which examined these validities using Campbell & Fiske's (1959) multitrait-multimethod paradigm.

Cognitive Response Indices

A second purpose of the present paper was to determine the extent to which differences between the independent Judges method and the self-rating method are accentuated or attenuated by the use of different cognitive response indices in the operationalism of cognitive response categories (Swasy 1980) . In particular, the present studies examined three commonly used indices the total number (gross frequency) of pro-, counter-, and neutral arguments emitted by the respondents; the net number (net frequency) of pro-, counter-, and neutral arguments emitted; and, the percentage of pro-, counter-, and neutral argumentation. The specific procedures used to compute these indices are described more fully by Swasy (1980) and in the method section of this paper.

Swasy ( 1980) noted that the meaningfulness of these and many other cognitive response indices has not been well developed. In a convincing empirical demonstration, he found that frequency indices ("weighted frequency counts" and "frequency counts") contained significant method variance, especially for counterargumentation and source derogation. On the other hand, little method variance was found with the relative proportion indicator for the three cognitive response constructs he examined (support arguments, counterarguments, and source derogation). Swasy (1980) concluded that the three cognitive response indices were valid but varied in their convergent and discriminant validities.

The present studies attempted to corroborate and extend Swasy's ( 1980) conclusions by examining three cognitive response indices (total, net, and percentage scores) of three cognitive response constructs (pro-, counter-, and neutral argumentation). Two of the three indices and two of the constructs are similar to the ones examined by Swasy ( 1980) . Unlike the present studies, however, Swasy (1980) did not include an examination of the convergent and discriminant validities of the independent Judges and the self-rating methods.

METHOD

Procedure

Study 1: "Forced Busing." Fifty-two business school students (22 females and 30 males, Md age = 23) enrolled in evening classes volunteered to participate in a study on "attitudes and current issues." The participants, who were run in groups of 10 to 20, were led to believe that they would be exposed to a videocassette tape of a recently delivered speech advocating forced busing. The topic was currently under heavy debate and was largely unpopular in the area in which most of the subJects resided. A VCR unit and monitor were in the room in full view to enhance the credibility of the cover storY.

The research participants were then told by a male experimenter that it would be helpful if they would first complete a three-part questionnaire before watching the tape. In the first part of the questionnaire, the participants were given two minutes to list any thoughts and ideas they had about the issue of busing. This thought-listing procedure was adapted from that of previous cognitive response research (e.g., Petty, Wells, & Brock 1976). In the second part of the questionnaire, the participants were asked to rate on 71-point graphic rating scales how favorable they were towards busing. Then, to obtain the self-rating method of cognitive response assessment, the third part of the questionnaire instructed the participants to go back over the thoughts they had previously listed and to rate each thought on a 71-point graphic rating scale according to how unfavorable/favorable it was towards busing.

Upon completion of the third and final part of the questionnaire, the participants were fully debriefed. None expressed suspicion or anger about the cover story and most seemed to be genuinely interested in the outcome of the study.

Finally, to obtain the independent Judges method of cognitive response assessment, two undergraduates (one female and one male) who were not involved as participants later rated each thought on the same 71-point scales used by the participants. The mean rating assigned to each thought by the two raters constituted the independent Judges score. Each rater had been given a brief 5-minute practice run Just prior to the actual rating task. Both were blind with respect to the design, hypotheses, and purpose of the study. Both raters worked independently of one another and, in fact, neither rater knew of the other's existence.

Study 2: "Tuition Fee Increase." Study 2 was conceptually similar to Study 1. In Study 2, however, 49 volunteers from upper level business school classes were first led to believe that they would have to write an essay on the tuition fee increases being proposed for their campus. The subjects then completed the three-part questionnaire measuring their cognitive responses and attitudes toward a tuition fee increase on their campus. (Arbitrarily, seven-point bipolar scales were used instead of the 71-point unipolar scales used in Study 1.) Instead of only two independent Judges, Study 2 used 25 independent Judges drawn from the 8 e population as the subjects. As in Study 1, these 25 Judges rated the thoughts without consulting one another.

Cognitive Response Traits

Three types of cognitive responses were treated as traits in the multitrait-multimethod examination: counterargumentation (the generation of cognitive responses unfavorable to the advocated position), proagumentation (the generation of cognitive responses favorable to the advocated position), and neutral-irrelevant argumentation (the generation of cognitive responses neutral or irrelevant to the advocated position). Operationally, those thoughts which were assigned a rating scale value that fell in lower half of the rating scale (in Study 1, those scores in the 0 to 34 range; in Study 2, from -3 to -1) were classified as counterarguments, those which fell in the upper half of the rating scale (in Study 1, 36 to 70; in Study 2, +1 to +3) were classified as proarguments, and those which received a mid-scale rating (in Study 1, mid-scale = 35; in Study 2, mid-scale = 0) were classified as neutral arguments. Neutral-irrelevant arguments were included in the present studies for completeness and because neutral arguments may prove to play an important role in low involvement information processing.

Dependent Measures of the Cognitive Response "Traits"

Total, net, and percentage scores for each type of argumentation were computed for each subject. All three of these scoring procedures are commonplace in the cognitive response literature, but they do not always yield similar results (e.g., Lammers 1979).

Total argumentation scores (gross frequency). Three total argumentation scores were computed for each subject: total counterargumentation (the total number of thoughts classified as counterarguments ), total proargumentation (the total number of thoughts classified as proarguments), and total neutral argumentation (the total number of thoughts classified as being neutral).

Net argumentation scores (net frequency). Each subject also received three net argumentation scores: net counterargumentation (the total counterargumentation score minus the total pro- and neutral argumentation scores), net proargumentation (the total proargumentation score minus the total counter- and neutral argumentation scores), and net neutral argumentation (the total neutral argumentation score minus the total pro- and counterargumentation scores). Each of these net argumentation scores represents a linear combination of thoughts yielding a composite, directional cognitive response activity index (Edell & Mitchell 1978; Wright 1973).

Argumentation percentage scores. Finally, the following three argumentation percentage scores were computed for each participant: counterargumentation percentage (the percentage of all thoughts classified as counterarguments), proargumentation percentage (the percentage of all thoughts classified as proarguments), and neutral argumentation percentage (the percentage of all thoughts classified as neutral).

RESULTS

Criterion-Related Validity

Criterion-related validity would be demonstrated by showing that counterargumentation was negatively related to attitudes toward the advocated topic, that proargumentation was positively related to attitudes toward the topic, and that neutral argumentation was unrelated to attitudes toward the topic. The relevant correlations between cognitive response scores and attitudes for both Study 1 and Study 2 are presented in Table 1. Both the self-rating method and the independent Judges method produced the necessary pattern of relationships between cognitive responses and attitudes.

Convergent Validity

Convergent validity refers to the extent to which different methods agree on their assessment of the same trait. In this study, convergent validity refers to the extent to which the self-rating and independent Judges methods agree on the measurement of counter-, pro-, and neutral argumentation.

TABLE 1

CRITERION-RELATED VALIDITY: CORRELATIONS WITH ATTITUDE SCORES

Monotrait-heteromethod coefficient. From a multitrait-multimethod approach, convergent validity is demonstrated by the monotrait-heteromethod correlations being significantly different from zero and sufficiently large to encourage further examination. In the multitrait (pro- v. counter- v. neutral argumentation) -multimethod (self-rating v. independent Judges) matrix presented in Tables 2 and 3 it can be seen that the convergent validity coefficients (the italicized correlations) were significantly different from zero for total pro-, counter-, and neutral argumentation (.81, .87, .24, respectively, in Study 1; .71, .60, and .54 in Study 2), for net pro-, counter-, and neutral argumentation (.84, .86, .75 in Study 1; .58, .55, .69 in Study 2), and for pro- and counterargumentation percentage scores (.84 and .85 in Study 1; .48 and .55 in Study 2). Unimpressive convergent validity coefficients of .18 (Study 1) and .26 (Study 2) were found for neutral argumentation percentage scores.

Monotrait-heteromethod means. Convergent validity can also be expressed by showing that the mean cognitive response scores obtained by one method are not significantly different from the mean scores derived from another method. The results of the correlated t-tests on the cognitive response scores are shown in Table 4. In Study 1, the pattern of the means indicates high convergence validity, though it is noteworthy that the outside Judges (M = 3.23) classified more of the thoughts as being counterarguments than did the subjects themselves (M = 2.96, 2 < .05). In Study 2, however, the independent Judges, relative to the subJects themselves, were more likely to classify the thoughts as proarguments and less likely to classify them as neutral arguments.

Discriminant Validity

Discriminant validity refers to the degree to which a trait can be differentiated from other traits. In the multitrait-multimethod approach, three criteria can provide evidence for discriminant validity (Campbell & Fiske 1959; Hubert & Baker 1979; Lawler 1967). First, a trait should correlate more highly with itself over two methods than with another trait using the same two methods. This involves computing the proportion of times that the monotrait-heteromethod coefficients are larger than the corresponding heterotrait-heteromethod coefficients (Ostrom 1969). For total argumentation scores that proportion was .92 (Study 1) and 1.00 ( Study 2) ; for net argumentation scores that proportion was .92 (Study 1) and .83 (Study 2); and, for argumentation percentage scores that proportion was .67 ( Study 1) and . 67 ( Study 2).

TABLE 2

STUDY 1: MULTITRAIT-MULTIMETHOD MATRIX

TABLE 3

STUDY 2: MULTITRAIT-MULTIMETHOD MATRIX

TABLE 4

MEANS OF COGNITIVE RESPONSES BY METHOD OF ASSESSMENT

Secondly, a variable should correlate higher with an independent effort to measure the same trait than with measures designed to get at different traits which happen to employ the same method. This involves computing the proportion of times that a trait's value in its validity diagonal is greater than its values in the heterotrait-monomethod triangles. For total argumentation scores, that proportion was found to be . 92 (Study 1) and 1.00 (Study 2); for net argumentation scores, the proportion was .67 (Study 1) and .67 (Study 2); and, for argumentation percentage scores, the proportion was .50 (Study 1) and .50 (Study 2).

A third way to assess discriminant validity is to show the same pattern of trait relationships in all of the heterotrait triangles (all triangles enclosed by either broken or solid lines). Such patterns were evidenced in the present studies. The correlations between proarguments and counterarguments were usually the largest correlations (in 12 of the 12 heterotrait triangles in Study 1 and in 10 of the 12 triangles in Study 2), and they were always negative. Interestingly, in Study 1 proargument measures were usually unrelated to the neutral argument measures (unrelated in 7 of 12 comparisons) but when related, were typically positively related. In Study 2, proargument measures were again usually unrelated to the neutral argument measures (unrelated in 10 of the 12 comparisons), but when related, were negatively related. As for the relationship between counterarguments and neutral arguments, Study 1 showed that for 6 of the 12 comparisons the relationship was not significant, and for the other 6 comparisons, the relationship was significantly negative. In Study 2, the only one significant correlation was found in the heterotrait triangles between counterarguments and neutral arguments, and it was a negative one. In sum, however, proarguments and counterarguments were not strongly related to the neutral argument measures.

DISCUSSION

The present two studies attempted to tar allY assess the validity of commonly used methods of cognitive response assessment. The similar pattern of results from these two studies showed that the self-rating method and the independent Judges method generally produced acceptable levels of criterion-related validity, convergent validity and discriminant validity. There were, however, some low points in the validity checks. For instance, the convergent validity coefficients for neutral argumentation, net neutral argumentation notwithstanding, were low. Perhaps this was artificially due to the low number of thoughts classified as neutral, or perhaps it was due to a genuine disagreement between actors and observers on what is or is not a neutral thought.

Since pro- and counterarguments represent opposite ends of an evaluative continuum, it would be expected that the discriminant validity would be high--and it was. However, discriminant validity was lowered by the use of percentage scores. This suggests that cognitive response studies may find total argument scores (gross frequency) to be more sensitive to treatment effects than percentage scores. If, however, the strength of a type of cognitive response relative to other types is of prime interest in a particular study, then the present results suggest a net argumentation scoring method may be a reasonably sound choice over a percentage scoring method.

Overall, it can be concluded that the widespread use of both methods of cognitive response assessment has probably not produced a breach of validity. Thus, more mundane concerns such as research cost and time can be allowed to more freely enter the research strategy with somewhat assuaged guilt feelings on the part of the researcher.

Finally, it is important to realize that a number of questions concerning the validity of cognitive response assessment were not addressed by the present studies. The most troublesome one concerns the validity of the thought-listing procedure itself, for if this procedure lacks construct validity, the specific concerns of the present study are irrelevant. On the other hand, it appears that recent social-psychological research (Cacioppo 1979, 1982; Cacioppo & Petty 1979b) has momentarily silenced questions concerning the validity of the thought-listing procedure itself.

The complete reference list of the approximately 100 cognitive response articles cited in this paper are available from the author.

REFERENCES

Belch, George E. (1981), "An Examination of Comparative and Noncomparative Television Commercials: The Effects of Claim Variation and Message Repetition on Cognitive Response and Message Acceptance," Journal of Marketing Research, 18 (August). 333-349.

Belch, George E. (1982), "The Effects of Television Commercial Repetition on Cognitive Response and Message Acceptance," Journal of Consumer Research, 9 (June), 56-65.

Belch, George E. and Michael A. Belch ( 1984 ), "An Investigation of the Effects of Repetition on Cognitive and Affective Reactions to Humorous and Serious Television Commercials," in Advances in Consumer Research, Vol. 11., ed. Thomas C. Kinnear, Provo: Association for Consumer Research 4-10.

Bettman, James R., Harold H. KassarJian, and Richard J. Lutz (1978), "Consumer Behavior," in Review of Marketing 1978, eds., G. Zaltman and Thomas V. Bonoma, Chicago: American Marketing Association.

Brock, Timothy C. (1967), "Communication Discrepancy and Intent to Persuade as Determinants of Counterargument Production." Journal of Experimental Social Psychology, 3, 296-309.

Cacioppo, John T. (1979), "Effects of Exogenous Changes in Heart Rate on Facilitation of Thought and Resistance to Persuasion," Journal of Personality and Social Psychology, 37, 489-498.

Cacioppo, John T. (1982) "Social Psychophysiology: A Classic Perspective and Contemporary Approach," Psychophysiology, 19 (3), 241-251.

Cacioppo, John T., Stephen H. Harkins, and Richard E. Petty (1981), "The Nature of Attitudes and Cognitive Responses and Their Relationships to Behavior," in Cognitive Responses in Persuasion, eds. Richard E. Petty, Thomas M. Ostrom, and Timothy C. Brock, Hillsdale, NJ: Lawrence Erlbaum.

Cacioppo, John T. and Richard E. Petty (1979a), "Effects of Message Repetition and Position on Cognitive Response, Recall, and Persuasion," Journal of Personality and Social Psychology, 37, 97-109.

Cacioppo, John T. and Richard E. Petty (1979b), "Attitudes and Cognitive Response: An Electrophysiological Approach," Journal of Personality and Social Psychology, 37, 2181-2199.

Cacioppo, John T. and Richard E. Petty (1980), "Persuasiveness in Communications is Affected by Exposure Frequency and Message Quality: A Theoretical and Empirical Analysis of Persisting Attitude Change," in Current Issues and Research in Advertising, eds., James H. Leigh and Claude R. Martin, Jr., Ann Arbor: University of Michigan Press.

Cacioppo, John T. and Richard E. Petty (1981)s "Social Psychological Procedures for Cognitive Response Assessment: The Thought-Listing Technique," in Cognitive Assessment, eds. Thomas V. Merluzzi, Carol R. Glass, and Hyles Genest, New York: Guilford Press, 309-342.

Cacioppo, John T., Richard E. Petty, and Leo R. Quintarar (1982), "Individual Differences in Relative Hemispheric Alpha Abundance and Cognitive Responses to Persuasive Communications," Journal of Personality and Social Psychology. 43 (September), 623-636.

Cacioppo, John T., Richard E. Petty, and Joseph A. Sidera ( 1982), "The Effects of a Salient Self-Schema on the Evaluation of Proattitudinal Editorials: Top-Down Versus Bottom-Up Message Processing," Journal of Experimental Social Psychology, 18, 324-338.

Calder, Bobby J. (1978), "Cognitive Response, Imagery, and Scripts: What is the Cognitive Basis of Attitude?" in Advances in Consumer Research, Vol. 5, ed. H. Keith Hunt, Ann Arbor, MI: Association for Consumer Research, 630-634.

Calder, Bobby J. and Robert E. Burnkrant (1977), "Interpersonal Influence on Consumer Behavior: An Attribution Theory Approach," Journal of Consumer Research. 4, 29-38.

Calder, Bobby J., Chester A. Insko, and Ben Yandell (1974), "The Relation of Cognitive and Memorial Processes to Persuasion in a Simulated Jury Trial," Journal of Applied Social Psychology, 4, 62-93.

Campbell, Donald T. and Donald W. Fiske ( 1950), "Convergent and Discriminant Validity by the Multitrait-Multimethod Matrix," Psychological Bulletin, 56, 81-105.

Carment, D. W., and Gillian Foster ( 1969), "The Relationship of Opinion-Strength and Order of Self-Produced Arguments to Number of Arguments Produced and on Change," Acta Psychologica, 31, 286-292.

Chaiken, Shelly (1980), "Heuristic versus Systematic Information Processing and the Use of Source versus Message Cues in Persuasion," Journal Personality and Social Psychology, 39, 752-766.

Chaiken, Shelly and Alice H. Eagly (1976), "Communication Modality as a Determinant of Message Persuasiveness and Message Comprehensibility," Journal of Personality and Social Psychology, 34, 605-614.

Cialdini, Robert B., Alan Levy, C. Peter Herman, Lynn T. Sozlowski, and Richard E. Petty (1976), "Elastic Shifts of Opinion: Determinants of Direction and Durability," Journal of Personality and Social Psychology, 34, 663-672.

Cialdini, Robert B., Richard E. Petty, and John T. Cacioppo ( 1981), "Attitude and Attitude Change," Annual Review of Psychology, 32, 357-404.

Cook, Thomas D. (1969), "Competence, Counterarguing, and Attitude Change," Journal of Personality, 37, 342-358.

Eagly, Alice H. ( 1974), "Comprehensibility of Persuasive Arguments as a Determinant of Opinion Change," Journal of Personality and Social Psychology, 29, 758-773.

Eagly, Alice H. and Rebecca Warren ( 1976), "Intelligence, Comprehension, and Opinion Change," Journal of Personality, 44, 226-242.

Edell, Julie A. and Andrew A. Mitchell (1978), "An Information Processing Approach to Cognitive Response," in Research Frontiers in Marketing: Dialogues and Directions. 1978 Educators' Proceedings, Chicago: American Marketing Association, 178-183.

Edell, Julie A. and Richard Staelin (1983), "The Information Processing of Pictures of Print Advertisement," Journal of Consumer Research, 10 (June), 45-61.

"Ford Has a Better Idea...Cognitive Response Measures to Improve TV Commercials," Marketing News, May 15, 1981, p. 23.

Greenwald, Anthony G. (1968), "Cognitive Learning, Cognitive Response to Persuasion, and Attitude Change," in Psychological Foundations of Attitudes, eds. Anthony G. Greenwald, Timothy C. Brock, and Thomas M. Ostrom, New York: Academic Press.

Harkins, Stephen G. and Richard E. Petty (1981a), "Effects of Source Magnification of Cognitive Effort on Attitudes: An Information-Processing View," Journal of Personality and Social Psychology, 40 (3), 401-413.

Harkins, Stephen G. and Richard E. Petty (1981b), "The Multiple Source Effect in Persuasion: The Effects of Distraction," Personality and Social Psychology Bulletin, 7 (4), 627-635.

Harmon, Robert R. and Kenneth A. Coney (1982), "The Persuasive Effects of Source Credibility in Buy and Lease Situations," Journal of Marketing Research, 19 (May), 255-260.

Harvey, John H., William John Ickes, and Robert F. Ridd (1976), New Directions in Attribution Research, Hillsdale, NJ: Lawrence Erlbaum.

Heesacker, Martin and Richard E. Petty (1981), "Effects of Recipient Posture on Persuasion," paper presented at the Midwestern Psychological Association, Detroit.

Heesacker, Martin, Richard E. Petty, and John T. Cacioppo (1983), "Field Dependence and Attitude Change: Source Credibility Can Alter Persuasion by Affecting Message-Relevant Thinking, " Journal of Personality. 51 (4), 653-666.

Herzberger, Sharon D. and Gerald L. Clore (1979), "Actor and Observer Attributions in a Multitrait-Multimethod Matrix," Journal of Research in Personality. 13, 1-15.

Hubert, Lawrence J. and Frank B. Baker (1979), "A Note on Analyzing the Multitrait-Multimethod Matrix: An Application of a Generalized Proximity Function Comparison," British Journal of Mathematical and Statistical Psychology, 32, 179-184.

Insko, Chester A., Allan E. Lind, and Stephen LaTour (1976). "Persuasion, Recall, and Thoughts," Representative Research in Social Psychology, 7, 66-78.

Insko, Chester A., William Turnbull, and Ben Yandell (1974), "Facilitative and Inhibiting Effects of Distraction on Attitude Change," Sociometry, 37, 508-528.

Jones, Edward E. (1979), "The Rocky Road From Act to Dispositions," American Psychologist, 34, 107-117.

Kassarjian, Harold H. (1982), "Consumer Psychology," Annual Review of Psychology, 33, 619-649.

Keating, John P. and Timothy C. Brock (1974), "Acceptance of Persuasion and the Inhibition of Counterargumentation Under Various Distraction Tasks," Journal of Experimental Social Psychology, 10, 301-309.

Kelman, Herbert C. (1953), "Attitude Change as a Function of Response Restriction," Human Relations, 6, 185-214.

Lammers, H. Bruce (1979), "An Empirical Examination of Self-Focused Attention Effects on Cognitive Responses to Advertising Stimuli," in Proceedings of the Southern Marketing Association, eds. Robert S. Franz, Robert M. Hopkins, and Allred G. Toma, Lafayette, LA: Southern Marketing Association, 289-292.

Lammers, H. Bruce (1982), "Effects of Biased Scanning and Distraction on Cognitive Responses," Journal of Social Psychology, 116, 99-105.

Lammers, H. Bruce (1983), "Balance Theory Effects of Corrective Advertising," paper presented at the meeting of the Western Psychological Association, San Francisco.

Lammers, H. Bruce and Lee A. Becker (1975), "The Interaction of Distraction, Choice, and Self-Esteem on Attitude Change,n paper presented at the meeting of the Midwestern Psychological Association, Chicago.

Lammers, B. Bruce and Lee A. Becker (1980), "Distraction Effects on the Perceived Extremity of a Communication and on Cognitive Responses," Personality and Social Psychology Bulletin, 6 (2), 261-266.

Lammers, H. Bruce, Laura Leibowitz, George Edw. Seymour, and Judith E. Hennessey (1983), "Immediate and Delayed Effects of Rumor on Cognitive Responses to Advertising Stimuli: A Trace Consolidation Approach," Journal of Business Research, 11, 173-185.

Lammers, H. Bruce, George Edw. Seymour, and Marilyn L. Wilkinson-Lammers (1981), "Source Expertise and Self-Focused Attention Effects on Biased Scanning of Advertisements," paper presented at the meeting of the American Psychological Association, Division of Consumer Psychology, Los Angeles.

Lawler, Edward E., III (1967), "The Multitrait-Multirater Approach to Measuring Managerial Job Performance," Journal of Applied Psychology, 51, 369-381.

Love, Robert E. and Anthony G. Greenwald (1978), "Cognitive Responses to Persuasion as Mediators of Opinion Change," Journal of Social Psychology, 104, 231-241.

Lutz, Richard J. and John L. Swasy (1977), "Integrating Cognitive Structure and Cognitive Response Approaches to Monitoring Communications Effects," in Advances in Consumer Research, Vol. 4, ed. William D. Perreault, Jr., Atlanta: Association for Consumer Research. 363-371.

Madden, Thomas J. and Kathleen Debevec (1983), "An Assessment of the Effects of the Time Provided f or Thought Elicitation on Subsequent Standard Scaled Measures of Attitudes and Intentions," in 1983 AMA Educators' Proceedings, eds. Patrick E. Murphy et al., Chicago: American Marketing Association, 415-419.

Marks, Larry J. and Jerry C. Olson (1981), "Toward a Cognitive Structure Conceptualization of Product Familiarity," in Advances in Consumer Research, ed., Kent B. Monroe, Ann Arbor: Association for Consumer Research.

McCullough J. Lee and Thomas M. Ostrom (1974), "Repetition of Highly Similar Messages and Attitude Change," Journal of Applied Psychology, 59, 395-397.

Miller, Norman and Robert S. Baron (1973), "On Measuring Counterarguing," Journal far the Theory of Social Behavior, 3, 101-118.

Mizerski, Richard W., Linda L. Golden, and Jerome Kernan (1979), "The Attribution Process in Consumer Decision Making," Journal of Consumer Research, 6, 123-140.

Olson, Jerry C., Daniel R. Toy, and Philip A. Dover (1978), "Mediating Effects of Cognitive Responses to Advertising on Cognitive Structure," in Advances in Consumer Behavior, Vol. 5, ed. H. Keith Hunt, Ann Arbor: Association for Consumer Research, 72-78.

Olson, Jerry C., Daniel R. Toy, and Philip A. Dover (1983), "Do Cognitive Responses Mediate the Effects of Advertising Content on Cognitive Structure?" Journal of Consumer Research, 9 (December). 245-262.

Osterhouse, Robert A. and Timothy C. Brock (1970), "Distraction Increases Yielding to Propaganda by Inhibiting Counterarguing," Journal of Personality and Social Psychology, 15, 344-358.

Ostrom, Thomas M. (1969), "The Relationship between the Affective, Behavioral, and Cognitive Components of Attitude," Journal of Experimental Social Psychology, 5, 12-30.

Percy, Larry and Martin R. Lautman (1981), "Linear Effects in Cognitive Response to Advertising," in Advances in Consumer Research, Vol. 8, ed. Kent B. Monroe, Ann Arbor: Association for Consumer Research.

Petty, Richard E. (1977), "The Importance or Cognitive Responses in Persuasion," in Advances in Consumer Research, Vol. 4, ed. William D. Perreault, Jr., Atlanta: Association for Consumer Research.

Petty, Richard E. and John T. Cacioppo (1979a), "Effects of Forewarning of Persuasive Intent and Involvement on Cognitive Responses and Persuasion," Personality and Social Psychology Bulletin, 5, 173-176.

Petty, Richard E. and John T. Cacioppo (1979b), "Issue Involvement Can Increase or Decrease Persuasion by Enhancing Message-Relevant Cognitive Responses," Journal of Personality and Social Psychology, 37 (October), 1915-1926.

Petty, Richard E. and John T. Cacioppo (1981), "Issue Involvement as a Moderator of the Effects on Attitude of Advertising Content and Context," in Advances in Consumer Research, Vol. 8, ed ., Kent B. Monroe, Ann Arbor: Association for Consumer Research.

Petty, Richard E. and John T. Cacioppo (1984), "The Effects of Involvement on Responses to Argument Quantity and Quality: Central and Peripheral Routes to Persuasion," Journal of Personality and Social Psychology, 46 (January), 69-81.

Petty, Richard E., John T. Cacioppo, and Martin Heesacker (1981), "Effects of Rhetorical Questions on Persuasion: A Cognitive Response Analysis," Journal of Personality and Social Psychology, 40 (3), 432-440.

Petty, Richard E., John T. Cacioppo, and David Schumann (1983), "Central and Peripheral Routes to Advertising Effectiveness: The Moderating Role of Involvement," Journal of Consumer Research, 10 (September), 135-146.

Petty, Richard E., Thomas M. Ostrom, and Timothy C. Brock (1981), Cognitive Responses in Persuasion, Hillsdale, NJ: Lawrence Erlbaum.

Petty, Richard E., Gary L. Wells, and Timothy C. Brock (1976), "Distraction Can Enhance or Reduce Yielding to Propaganda: Thought Disruption Versus Effort Justification," Journal of Personality and Social Psychology, 34 (November), 874-884.

Ray, Michael L. (1977), "When Does Consumer Information Processing Research Actually Have Anything To Do With Consumer Information Processing?" in Advances in Consumer Research, Vol 4., ed. William D. Perreault, Jr., Atlanta: Association for Consumer Research, 372-375.

Roberts, Donald F. and Nathan Maccoby (1973), "Information Processing and Persuasion: Counterarguing Behavior," in New Models for Mass Communication Research, ed. Peter Clarke, Beverly Hills: Sage, 269-307.

Romer, Daniel (1979a), "Distraction, Counterarguing and the Internalization of Attitude Change," European Journal of Social Psychology, 9, 1-17.

Romer, Daniel (1979b), "Internalization Versus Identification in the Laboratory: A Causal Analysis of Attitude Change," Journal of Personality and Social Psychology, 37, 2171-2180.

Ross, L. (1977), "The Intuitive Psychologist and His Shortcomings: Distortions in the Attribution Process," in Advances in Experimental Social Psychology, Vol. 10, ed. Leonard Berkowitz, New York: Academic Press.

Shavitt, Sharon and Timothy C. Brock (1984), "Consumer Research Validity: The Effect of Social Settings on Cognitive Responding to Television Commercials," in Advances in Consumer Research, Vol. 11, ed. Thomas C. Kinnear, Provo: Association for Consumer Research, 18-23.

Smith, Robert E. and Shelby D. Hunt (1978), "Attributional Processes and Effects in Promotional Situations," Journal of Consumer Research, 5, 149-158.

Sternthal, Brian, Ruby Dholakia, and Clark Leavitt (1978), "The Persuasive Effect of Source Credibility: Tests of Cognitive Response," Journal of Consumer Research, 4 (December), 252-260.

Swasy, John T. (1980), "A Multitrait-Multimethod Analysis of Cognitive Response Indicators by Analysis of Covariance Structures," in Marketing in the 80's: Chances and Challenges, eds., Richard P. Bagozzi et al., Chicago: American Marketing Association, 362-365.

Swinyard, William R. (1981), "The Interaction Between Comparative Advertising and Copy Claim Variation," Journal of Marketing Research, 18, 175-186.

Toy, Daniel R. (1982), "Monitoring Communication Effects: A Cognitive structure/Cognitive Response Approach," Journal of Consumer Research, 9 (June), 66-70.

Tyebjee, Tyzoon T. (1978), "Cognitive Response and the Reception Environment of Advertising," in Research Frontiers in Marketing: Dialogues and Directions. 1978 Educators' Proceedings, ed. Subhash C. Jain, Chicago: American Marketing Association, 174-177.

Wells, Gary L. and Richard E. Petty (1980), "The Effects of Overt Head Movement on Persuasion: Compatibility and Incompatibility of Responses," Basic and Applied Social Psychology, 1 (September), 219-230.

Wilson, R. Dale and Aydin Muderrisoglu (1980), "An Analysis of Cognitive Responses to Advertising," in Advances in Consumer Research. Vol. 7.

Wright, Peter L. (1973), "The Cognitive Processes Mediating Acceptance of Advertising," Journal of Marketing Research, 10, 53-62.

Wright, Peter L. (1974), "Analyzing Media Effects on Advertising Responses," Public Opinion Quarterly, 38, 192-205.

Wright, Peter L. (1975), "Factors Affecting Cognitive Resistance to Advertising," Journal of Consumer Research. 2, 1-9.

Wright, Peter L. (1980), "Message-Evoked Thoughts: Persuasion Research Using Thought Verbalizations," Journal or Consumer Research, 7 (September), 151-175.

Wright, Peter L. and Peter D. Rip (1981), "Retrospective Reports on the Causes of Decisions," Journal of Personality and Social Psychology 40, 601-614.

----------------------------------------