Some Measurement Issues in Consumer Research

Citation:

Ved Prakash (1992) ,"Some Measurement Issues in Consumer Research", in NA - Advances in Consumer Research Volume 19, eds. John F. Sherry, Jr. and Brian Sternthal, Provo, UT : Association for Consumer Research, Pages: 813-816.

Advances in Consumer Research Volume 19, 1992      Pages 813-816

SOME MEASUREMENT ISSUES IN CONSUMER RESEARCH

Ved Prakash, Morgan State University

INTRODUCTION

The purpose of this paper is to discuss the issues raised by the three papers accepted by the Association for Consumer Research for presentation in the session on "Measurement Issues". First, a broad criteria for discussion of measurement issues is presented and then each of the papers is discussed. This discussion concludes with suggestions for applications of some of the concepts presented in one of the papers. The papers under discussion are as follows:

1) Gabriel S. Haberland and Peter A. Dacin, "The Development of a Measure to Assess Viewers Judgements of the Creativity of an Advertisement: A Preliminary Study".

2) Paul L. Sauer, Peter R. Dickson and Kenneth A. Lord, "A Multiphase Thought Elicitation Coding Scheme for Cognitive Response Analysis".

3) Jonathan E. Brill, "Scales to Measure Social Power in a Consumer Context".

To begin with, it would be useful to lay down some criteria for evaluation of measurement issues. For a sound measurement of constructs in consumer research three criteria may be useful. First, the purpose of the research problem ought to be meaningful and significant; the construct being developed should be conceptually sound. Secondly, the constructs that are developed ought to be rigorously measured. The measures should meet the tests of reliability and validity. Special attention needs to be paid to the requirements of validity suggested by Campbell and Fiske (1959). Reliability is also an important consideration that is often ignored. Peter (1979) reported that only 5% of the studies assessed the reliability of the measures employed. Third, for the purposes of consumer research it would be desirable to show relevant applications with some concrete examples preferably with empirical data. These applications should show effectiveness of the measures in a real life situation different from the one used in an experimental setting for testing the constructs developed. With these guidelines in view we may now proceed to discuss the three papers mentioned above.

Haberland and Dacin Paper

Haberland and Dacin's paper is very clear in its purpose i.e. to develop a valid construct of creativity in advertising from viewer's point of view. The authors make some new contribution to the consumer behavior literature. The concept of creativity has long been researched in the psychological literature, and has been intuitively practiced by the advertising industry, but it has not been objectively tested in an advertising setting. The April 15, 1991 issue of the Marketing News has many articles on the concept of advertising creativity from the practitioner's point of view. Dacin and Haberland have drawn extensively from the literature in various fields: Advertising (Hinderyckx 1991, Trout and Ries 1989) and Psychology (MacKinnon 1962, Bruner 1962, Jackson and Messick 1965). Based on these studies, Haberland and Dacin deduce four dimensions for measurement of creativity in advertising i.e. 1) originality or innovativeness, 2) meaningfulness, 3) reformulation or modification of consumer attitudes, and 4) condensation i.e. simplicity in packing a lot of information in a few words; parsimony could be another word for it especially considering the time compression and high rates of inserting advertisements in the media.

Haberland and Dacin have done a satisfactory job of collecting 42 items relating to creativity and reducing them to 16 purified items. For these 16 items, the reliabilities as reflected in the coefficient alphas are acceptable e.g. at the overall level (.92), and for the individual dimensions, .87 for originality, .77 for meaningfulness, .77 for reformulation and .66 for condensation. These four dimensions are confirmed by the results of factor analysis as indicated in Table 2. The assessment of convergent and discriminant validity on the multitrait-multimethod matrix (MTMM) as suggested by Campbell and Fiske (1959) and as displayed in Table 3 is well done except that for divergent validity the choice of the divergent measures humor and attitude toward the brand is debatable. Conceptually, humor can be an essential part of a creative advertisement and therefore the two constructs are not independent. Also some of the items that are used to measure attitude toward the brand can also be included in the measure of creativity of advertisement. There is a further question regarding the reliabilities of Method 2 in the MTMM where single-item 10 point scales are employed (as different from multi-item Likert and Semantic Differential scales in method 1). Psychometrically, single item scales have been found to be less reliable than those based on multiple items (Nunnally 1967; Peter 1979, p. 16).

Evaluation of Haberland and Dacin paper on the third criterion that I mentioned at the outset led me to some disappointment. Even though the authors have developed a valid measure of creativity, they have not reported any empirical results obtained from a real life situation to test the effectiveness of this measure. Their claim that, "while creativity is currently measured by advertising professionals and brand managers, our measure focuses on viewers' judgements of the creativity of the advertisements", is an exaggeration in the absence of an evaluation of the effectiveness of this measure in a real life setting. It is hoped that the authors would be doing further research into the real life application of this measure of creativity and reporting results in the literature.

Sauer, Dickson and Lord Paper

Sauer, Dickson, and Lord's paper entitled, "A Multiphase Thought Elicitation Coding Scheme for Cognitive Response Analysis", has a well defined purpose and is based on a sound review of the existing literature. The main purpose of their paper is to improve upon an earlier coding scheme developed by Dickson and Sauer (1987). This improvement is accomplished by incorporating the suggestions of Brucks et al. (1988) with respect to the categorization of cognitive thoughts. The study of verbal protocols in marketing was pioneered by Wright (1973) who suggested three main categories for dividing verbal thoughts: support arguments, counterarguments and source derogation. Bettman and Park (1980) further refined the method of analysis and division of verbal protocols according to brand, advertisement etc. Cacioppo, Harkins and Petty (1981) divided verbal thoughts into three major categories: Target, Origin and Polarity. Brucks et al. (1988) added a fourth dimension i.e. Relevance to the three dimensions suggested by Cacioppo et al. (1988). Brucks et al. tested the application of their coding scheme to determine the cognitive structures of children's responses to television advertising.

In the current paper Sauer et al. seek validation of their scheme of coding verbal thoughts (toward the choice of a Sony Compact Disk player). These authors have done a good methodological analysis by classifying verbal responses into four categories: Type (Fishbein - Ajzen 1975, Unidimensional Tripartite theory), Target of Thoughts (Cacioppo, Harkins and Petty 1981), Self-Relevance of Thought (Markus 1980) and Polarity of Thoughts (Cacioppo, Harkins and Petty 1981). Tables 4, 5, 6 in their paper indicate the crosstabulations of the responses into various categories i.e. Target by Polarity, Target by Type, and Target by Self-Relevance. There is a good deal of similarity in the results obtained with those of Brucks et al. (1988). These results indicate associative links in the pattern of thinking in cognitive processing. Sauer et al. go a step further by aggregating response dimensions and applying the McKenzie et al. (1986) cognitive structure model to the experimental data. The results show that Type and Relevance are the predominant categories in their research data.

With respect to real life applications of this scheme, I would look forward to any results that might be reported by these authors in future. In my overall evaluation, this study was too methodologically driven and less space was devoted to conceptual explication to real life consumer and advertising issues.

Brill's Paper on Social Power

Brill's paper entitled, "Scales to Measure Social Power in a Consumer Context", attempts to construct a scale to measure the concept of social power. In this paper social power is defined as the degrees of "influence" and "resistance". Brill concludes, "It has been argued that power is an attributed outcome deriving from a social actor's dependency on another within a specific context or social situation. Specifically, it is dependency not power, that has been posited to influence interpersonal behavior". A reading of the literature showed me that the topic of power has been extensively researched in the retailing literature in the context of inter and intra-channel relationships ((see Gaski (1984) for a review of the extant literature on channel power, conflict and contravailing power; El-Ansary and Stern (1972); Etgar 1978; Lusch, and Brown ((1982)). This literature in turn was based heavily on Social Psychological literature ((see Dahl (1957), where power has been defined as "the ability to evoke a change in another's behavior"; French and Raven (1959) identify five sources of power in terms of rewards, penalties, expertise, referent and identification; Thibaut and Kelley (1959) define power as, "an individual's power over another derives from the latter's being dependent upon him")).

Even though Brill has made a good and well-intentioned effort to measure "the concept of social power in a consumer context", I am a bit disappointed. First, from a conceptual point of view not enough explanation has been provided on the significance of the concept of social power and how this concept is different from the existing concepts being utilized in consumer behavior literature. Only a few lines in the second paragraph in the introduction section provide some justification for a study of social power in consumer behavior where Brill States, "yet, research emphasizing power relationship between retailers and consumers, the buyers and sellers at the end of many marketing channel systems, is relatively uncommon. And, even when these power relationships are considered, the focus is typically on the retailer with little, if any, regard paid to power associated with consumers". It would have been more desirable to elaborate on the importance of social power in a variety of consumer settings and to develop a model and suggest some hypotheses in a given context. The concept of social power is not new in consumer behavior. Fishbein and Ajzen's (1975) extended model of behavioral intention is relevant here, especially the concepts of social norms and motivation to comply. The latter concept is similar to the idea of social resistance mentioned by Brill. There is a whole lot of related literature on reference group, personal values, attributions, consumer complaint behavior (CCB) from where information on social power and related concepts could have been drawn. Instead, Brill uses a lot of space in the paper on the discussion of the definitions of power as found in the psychological literature. Perhaps, Brill should have presented a conceptual model in consumer context. It would have been better for Brill to expand on the interdependent relationship between the retailer and the consumer. Even though Brill tries to develop a general concept of social power, he does not seem to rise above the retailing context. There seems to be some confusion whether the intended context of Brill's future application is retailer-consumer relationship or some other possibilities in consumer research e.g. consumer complaint behavior. This same conceptual flaw is evident in the discussion of the potential applications of the measure of social power in the section of the paper on research applications. More elaborate, imaginative, treatment could have been given to potential applications.

Let us now evaluate this paper from a methodological point of view. For operational purposes the construct of "Consumer Power" (or social power) is broken down into four sub-scales: Social Influence (SI), Social Resistance (SR), Consumer Influence (CI), and Consumer Resistance (CR). Conceptually social resistance and consumer resistance are not the same, just as social resistance and consumer resistance are not the same. When we look at the "consumer power" and "consumer resistance" items given in the appendix to the paper, many of the items can also be found in a scale of consumer discontent (Lundstrom and Lamont 1976) or a scale of consumer alienation (Allison 1978). In his paper, Brill does not clarify how the pool of (39) items for his study was developed for these four subscales mentioned above. Brill, however, does mention the source of two other scales used for validity check i.e. Powerful Others Control (POC) and Internal Control (IC); for these 2 scales, 24 items were borrowed from Levenson (1974) who in turn based them on Rotter's (1966) Internal-External Locus of Central Construct. Alpha reliability coefficients for SI (.87), SR (.62), CI (.66), CR (.75) were reported. Relatively low reliabilities for SR and CI are justified by Brill on the ground that each scale has the desirable characteristic of having items with large standard deviations. This reasoning along with the conclusion that "these two scales would seem to offer high discriminatory power" is not very convincing. The analysis of convergent and discriminant validity following the display of Table 2 is very confusing. This analysis did not seem to rigorously follow the guidelines suggested by Campbell and Fiske (1959) by the multitrait-multimethod matrix (MMTM). A better presentation and discussion of the validity analysis would have been desirable.

In the discussion of limitations of the study, Brill seems to think that having a convenience sample is a serious limitation. This is not necessarily true. As Calder, Phillips and Tybout (1981) point out that if the purpose of a study is theory development and testing of hypotheses, convenience sample or a laboratory study is preferable to a representative sample or a field study (where the goal is generalizability). Since theory development is the main purpose of Brill's paper, convenience sample is not a serious limitation. It is the other conceptual and methodological weaknesses pointed out here that Brill should be concerned with.

Finally, as I mentioned earlier, in the area of giving concrete examples of potential applications of the scales developed, Brill's paper could have been more imaginative. I will now present one example of my own for a potential application of the concepts of consumer influence and consumer resistance in the area of consumer complaint behavior (CCB). The CCB literature shows that there are three main methods of expressing complaints: Voice, Private Party and Third Party (Singh 1988). The main antecedents CCB are: the intensity of dissatisfaction, attitude toward complaining, expectancy-value of complaining, and power experience complaining. We could add one more antecedent to the CCB i.e. consumer influence - consumer resistance, i.e. how much leverage the consumer has over the retailer or the manufacturer in getting redress for the complaint. This is especially true for voicing and third party action. The resolution of consumer complaints could be a net result of the relative power of the two parties i.e. the seller and the consumer. Also, there seems to be some overlap between the items included in the consumer resistance construct developed by Brill and the attitude toward the act of complaining widely used in CCB literature (Richins 1983a, 1983b; Singh 1990). Perhaps, a judicious combination of items from the two scales could be very useful to the multidimensional conceptualization of the CCB.

CONCLUSION

I have discussed three papers on the three useful criteria: a) the conceptual soundness and significance of the construct being measured, b) methodological rigor in applying tests of reliability and validity, c) the presentation of empirical data to show some real life application of the constructs developed. All the three papers have made useful contribution to consumer research e.g. Haberland and Dacin in the area of creativity in advertising, Sauer et al. in developing a coding scheme for cognitive thoughts and Brill on social and consumer power. All the three papers were however deficient on the third criterion I laid out i.e. empirical data on some real life application. I hope that they would be doing so in future. I found Haberland and Dacin paper and Sauer et al. particularly sound conceptually and methodologically. I found Brill's paper a little weak conceptually and methodologically. It is hoped that these suggestions would be useful for researchers in the area.

REFERENCES

Allison, Neil K. (1978), "A Psychometric Development of a Test for Consumer Alienation from the Marketplace", Journal of Marketing Research, Vol. XV (November 1978), 565-75.

Bruner, Jerome S. (1962), "The Conditions of Creativity", in Howard E. Gruber, Glenn Terrell and Michael Werthheimer (eds.) Contemporary Approaches to Creative Thinking, New York: Atherton Press.

Brucks, Merrie, Gary M. Armstrong and Marvin E. Goldberg (1988), "Children's Use of Cognitive Defenses Against Television Advertising: A Cognitive Response Approach", Journal of Consumer Research, 14:4 (March), 471-482.

Cacioppo, John T., Stephen G. Harkins and Richards E. Petty (1981), "The Nature of Attitudes and Cognitive Responses and Their Relationship to Behavior", in Richard E. Petty, Thomas M. Ostrom and Timothy C. Brock (eds.) Cognitive Responses in Persuasion, Hillsdale, New Jersey: Lawrence Erlbaum.

Calder, Bobby J., Lynn W. Phillips and Alice M. Tybout (1981), "Designing Research for Application", Journal of Consumer Research, Vol. 8, No. 2 (September), 197-207.

Campbell, Donald T. and Donald W. Fiske (1959), "Convergent and Discriminant Validation by he Multitrait-Multimethod Matrix", Psychological Bulletin, 56, 81-105.

Dahl, Robert A. (1957), "The Concept of Power", Behavioral Science, 2 (July), 201-218.

Dickson, Peter R. and Paul L. Sauer (1987), "Copy Testing, Thought Elicitation and Attitude Theory: Two's Company, Three's a Crowd?", Advances in Consumer Research, Vol. 14, Association for Consumer Research, Provo: Utah.

El-Ansary, Adel I. and Louis W. Stern (1972), "Power Measurement in the Distribution Channel", Journal of Marketing Research, 9 (February), 47-52.

Fishbein, Martin and Icek Ajzen (1975), Belief, Attitude, Intention and Behavior, Reading, Massachusetts: Addison-Wesley.

French, John R. Jr. and Bertram Raven (1959), "The Bases of Social Power", in Darwin Cartwright (ed.) Studies of Social Power, Ann Arbor: University of Michigan Press.

Gaski, John F. (1984), "The Theory of Power and Conflict in Channels of Distribution" Journal of Marketing, 48 (Summer), 9-29.

Hinderyckx, Greg (1991), "Experts Clash over Creativity", Marketing News, 25 (April 15, 1991), p.8.

Jackson, Phillip W. and Samuel Messick (1965), "The Person, the Product and the Response: Conceptual Problems in the Assessment of Creativity", Journal of Personality, 33, 309-329.

Levenson, Hanna (1974), "Activitism and Powerful Others: Distinctions within the Concept of Internal-External Control", Journal of Personality Assessment, 38 (August), 377-383.

Lundstrom, William J. and Lawrence M. Lamont (1976), "The Development of a Scale to Measure Consumer Discontent", Journal of Marketing Research, 12 (November), 373-81.

Markus, Hazel (1980), "The Self in Thought and Memory", in D.M. Wegner and R.R. Vallacher (eds.), New York: Oxford Press, 102-130.

McKenzie, Scott B., Richard J. Lutz and George E. Belch (1986), "The Role of Attitude Toward the Ad as a Mediator of Advertising Effectiveness: A Test of Competing Explanations", Journal of Marketing Research, XXIII (May), 130-143.

MacKinnon, Donald (1962), "The Nature and Nurture of Creative Talent", American Psychologist, 17, 484-95.

Nunnally, Jum C. (1978), Psychometric Theory, 2nd ed., New York: McGraw-Hill.

Peter, J. Paul (1979), "Reliability : A Review of Psychometric Basics and Recent Marketing Practices", Journal of Marketing Research, 16 (February), 6-17.

Richins, Marsha (1983a), "Negative Word-of-Mouth by Dissatisfied Consumers: A Pilot Study", Journal of Marketing, 47 (Winter), 68-78.

Richins, Marsha (1983b), "An Analysis of Consumer Interaction Style in the Marketplace", Journal of Consumer Research, 10 (June) 73-82.

Rotter, Julian B. (1966), "Generalized Expectancies for Internal Versus External Control of Reinforcement", Psychological Monographs: General and Applied, 80 (No. 1, Whole No. 609), 1-28.

Singh, Jagdip (1988), "Consumer Complaint Intentions and Behavior: Definitional and Taxonomical Issues", Journal of Marketing, Vol. 52 (January 1988), 93-107.

Singh, Jagdip (1990), "A Typology of Consumer Dissatisfaction Response Styles", Journal of Retailing, Vol. 66, No. 1 (Spring), 57-99.

Thibaut, John W. and Harold H. Kelley (1959), The Social Psychology of Groups, New York: John Wiley and Sons.

Trout, Jack and Al Ries (1989), "The Decline and Fall of Advertising", Advertising Age, 60 (June 26), p. 20.

Wright, Peter L. (1973), "The Cognitive Processes Mediating Acceptance of Advertising" Journal of Marketing Research, Vol. 10, 53-62.

----------------------------------------