A Potpourri of Consumer Research Methods

Citation:

Terence A. Shimp (1983) ,"A Potpourri of Consumer Research Methods", in NA - Advances in Consumer Research Volume 10, eds. Richard P. Bagozzi and Alice M. Tybout, Ann Abor, MI : Association for Consumer Research, Pages: 127-129.

Advances in Consumer Research Volume 10, 1983      Pages 127-129

A POTPOURRI OF CONSUMER RESEARCH METHODS

Terence A. Shimp, University of South Carolina

INTRODUCTION

The consumer behavior discipline has experienced in recent years a rapid evolution in the application of research methods, and probably has attained the growth stage in its methodological maturation. New and useful methods are borrowed from cognate disciplines, refined (sometimes), and rapidly diffused. Established methods are challenged, scrutinized, and ultimately improved or eliminated. Occasionally new methods are even developed within the discipline.

Advancement of the discipline requires continual appraisal and reappraisal of these research methods. Such a reappraisal is undertaken by two of the papers in this session. Fern offers a critique of focus group methodology and provides interesting suggestions for future research. The CAD scale is examined once again by Tyagi, who replicates Noerager's (1979) earlier effort to assess the scale's validity. The paper by Dillon and Madden is an innovative undertaking that truly captures the spirit of "new methods development" alluded to above. Their work is particularly noteworthy by its ambitious objective of explicitly modeling measurement error and methods variance into measurement scales constructed from discrete data.

Though common threads, however thin, can always be identified, the only substantive commonality among the three papers in this session is that all involve research methods used (or potentially usable) by commercial and academic consumer researchers. With this in mind, the papers are discussed separately and specific suggestions for future research and implications-for the discipline are made where appropriate. The ordering of papers is along a qualitative-quantitative dimension, with Fern's focus group research discussed first and Dillon and "Madden's scaling model last.

SCIENTIFIC RESEARCH ON FOCUS GROUPS-A HALL OR MIRRORS?

The use of focus groups is firmly entrenched in commercial consumer research, representing an empirical regularity often noted with practical research methods-after rapid diffusion in the commercial research community, research practitioners and members of the scientific community eventually challenge whether the methods are the panaceas they often are claimed to be.

This is evident in the case of focus groups with criticism from practitioners (e.g., Buncher 1982) and from academia with insightful critiques such as Calder's (1977). Fern's work, both here and in his recent JMR article (Fern 1982), represents the most recent statement on the use of focus groups. His major contribution is by way of offering empirical documentation of previously untested conventional wisdoms.

Ac the sake of severe oversimplification, Fern's thesis may be stated as follows: the use of focus groups is based on a variety of questionable assumptions (concerning the superiority of focus interviews over individual interviews, the ideal group size, etc.) and some of these assumptions simply do not hold up under empirical scrutiny. For example, Fern (1982) found that a greater number and higher quality or ideas were generated by individuals than by focus groups. One might quibble with certain aspects of Fern's Research (e.g., comparisons between groups of size four and eight when actual focus groups generally include eight or more people), but the fact remains that his arguments are keen and his research methods sound.

I have chosen in the comments that follow to raise some philosophical issues about the fruitfulness of future scientific research along the lines initiated by Fern (1982) and advocated by him in these proceedings. [Scientific research is used here in the sense of research that is directed at the ultimate goal of understanding and explaining consumer behavior phenomena. Commercial research, by contrast, is concerned with the more immediate objective of predicting specific outcomes. The distinction is not intended to be pejorative, but simply to indicate that all research is not the same. The point of the distinction is important for the subsequent evaluation of Fern's work.] The comments are concluded with suggestions for additional research, but research that follows a different philosoPhical course than that advocated by Fern.

Like Nailing Jello to a Tree

The one thought that pervaded my thinking in preparing these remarks was that the goal of scientifically identifying optimum focus group procedures and methods is probably as hopeless as the proverbial task of nailing jello to a tree. Attempting to arrive at an empirical conclusion regarding such issues as what group size is ideal and whether a moderator should be present is complicated, perhaps hopelessly, by the inherent idiosyncracy of focus group practice. What is "best" would seem entirely contingent on the particular set of research objectives; on the psycho-physical circumstances under which a focus group is conducted; the intellectual, interpersonal, and verbal skills of the moderator; the social, personality, verbal, and intellectual attributes or participants; and, no doubt, on other factors as well. Further muddling the evaluation is the fact that what is best or optimum in focus group practice is crucially dependent on the specific objective of a particular focus group, which, in turn, depends on the purpose of the study (hypothesis generation, concept development, etc.) as well as on the particular needs of the research user.

This philosophy of despair does not intend to suggest that the scientific community should never scrutinize practical research methods (consider for example the many contributions from the academic community in conjoint analysis research). What is being suggested instead is that the nature of focus groups is so terribly idiosyncratic (for the reasons elaborated above) that it virtually negates the value of what must inevitably be limited and sporadic evaluation efforts from the scientific community. I am reminded of Cronbach's pessimism as he commented on the state of scientific psychology and the role of interaction effects: "Once we attend to interactions, we enter a hall of mirrors that extends to infinity. However far we carry our analysis--co third order or fifth order or any order--untested interactions of a still higher order can be envisioned" (p. 119).

Recommended Research

The practice of "focus grouping" is an important form of consumption undertaken by numerous organizational buyers, thereby justifying research by the scientific community. What I suggest, however, is a different type of research than the evaluation activity pursued by Fern (1982) and recommended in his preceding paper. (This in no way intends to disparage Fern's thoughtful and serious efforts )

Because description and classification are first steps in knowledge generation, good old-fashioned exploratory research is sorely needed. The need for such research is perhaps best illustrated by the fact that no consensus exists on a matter as basic as the typical size of focus groups. For example, Calder (1976) says the typical size is 8-10; Cox et al. (1976) assert that it is 8-12. Other issues are these: Why is the use of focus groups so widespread, and what considerations influence the choice of focus groups over alternative methodologies? How are focus group findings incorporated into corporate decision making? How often are focus group results incorporated into subsequent "quantitative" research endeavors? What do focus group participants think about the exercises, are they satisfied, have they had sufficient opportunity to articulate their views?

These and a host of other questions need to be addressed in studying the practice of focus groups. This form of research would advance knowledge by describing and ultimately explaining an important form of consumption. Evaluation research, by comparison, is inherently limited in its ability to enhance knowledge of consumer behavior. In the same vein as Brunswik (1956), I would argue that generalizations about consumer behavior require representative samples of situations as well as of subjects. I am dubious that this will happen with the type of research recommended by Fern (1982).

THE CAD SCALE REVISITED

[Appreciation is extended to Joel Cohen for his thoughtful remarks in response to questions about his CAD scale development work and directions for future research. I, of course, am solely responsible for any unpardonable sins committed hereafter.]

Joel Cohen's innovative work on the CAD scale was published fifteen years ago (Cohen 1967). The intervening years have witnessed increasing sophistication in the scale development work in marketing and consumer behavior, but this initial work remains an important contribution to the discipline--albeit one more important perhaps in form than in substance. Research to date, including the preceding paper bs Tyagi, has not been particularly favorable toward the CAD scale in terms of convergent/discriminant validity (e.g., Heeler and Ray 1979; Noerager 1979) nor criterion validity (Woodside and Andress 1976).

Nevertheless, as will be argued subsequently, the validation efforts have not been flawless and, perhaps most important, the use of the CAD scale to attempt to explain product and brand choice behavior has probably been misguided and incompatible with the potential role of interpersonal orientation as an explanation of consumer behavior.

A few comments regarding Tyagi's research are in order before entertaining these more general issues. The major problem with Tyagi's study is the use of Stern's (1970) nurturance, aggression, and autonomy traits as alternate methods for measuring the CAD dimensions and for assessing convergent and discriminant validities in an MT-MM framework. The presumption in so doing is that these traits are equivalent to the compliance, aggression, and detached traits. Aside from the apparent similarity in the aggression trait, isomorphism between the other two is open to conjecture. An absence of perfect matching amounts to mixing apples with oranges, traits with measures.

In other words, Tyagi's test may have inextricably confounded traits with methods, thereby vitiating Campbell and Fiske's (1959) advice to use maximally different methods to measure the same traits. Instead, Tyagi has used very similar methods to measure what may be different traits. (Note: Tyagi didn't arbitrarily select these traits but appropriately relied on the advice of three personality psychologists. However, a review of the evidence in Noerager's (1979) research, who used a similar procedure, will show that expert psychologists may be limited in their ability to correctly match personality traits.)

Another criticism of Tyagi's research relates to his use of conventional, exploratory factor analysis to assess the factor structure of the CAD scale, when a confirmatory factor analysis would have provided a more rigorous test and a better indication of both the essential dimensions in the CAD scale and the specific items that may be expendable due to unreliable measurement.

A final point about Tyagi's paper is that virtually no direction for future research is offered. One would minimally expect specific suggestions and directions regarding future testing and application of the CAD scale. I will attempt to pg up where Tyagi left off.

Where to From Here?

Testing flaws aside, the evidence in support of the CAD scale's validity has not been particularly encouraging. This, however, does not permit the conclusion that the scale is valueless. To the contrary, the scale is theoretically meaningful (Bagozzi 1980, pp. 114-191) and therefore worthy of additional investigation. Better tests of convergent and discriminant validity are needed, and efforts to assess nomological validity are essential. Validation efforts in consumer behavior and marketing have perhaps overemphasized measurement validity considerations and have not devoted sufficient attention to theoretical relations among constructs.

My major recommendation is that future CAD research absolutely not attempt to use the CAD scale to predict specific product or brand choice behavior. Past efforts have basically been unsuccessful in this attempt (Cohen 1967; Woodside and Andress 1976). The reason in retrospect is clear: personality traits are incapable of accounting for much variability in specific acts of consumer choice behavior (Kassarjian 1971). Kernan (! 977) in a different context perhaps summed it up best in saying the same product can be consumed by different consumers, in different ways, for different reasons. Rather that attempting to predict specific purchase acts, applications of the CAD scale should be directed at attempts to account for differences in classes of behavior. Such an objective is compatible with Ajzen and Fishbein's (1980) similar position that the attitude construct is best used to explain a general class of related behaviors rather than specific acts.

Consumers with different interpersonal orientations should indeed be expected to differ in certain general consumption behaviors, namely those that specifically involve interpersonal relations. The CAD scale may be of particular value in explaining information search and store choice, as interpersonal relations are intertwined with these behaviors. For example, it might be expected that compliant individuals would be most likely to engage in catalog shopping, because this impersonal form of shopping enables the compliant personality a means of avoiding the tendency to yield to personal sales pressures.

In summary, the CAD scale still holds promise as a useful instrument for consumer researchers. However, future research must go beyond tests of simple factor structure, convergent validity, etc.; must test theoretical predictors based on interpersonal orientation theory; and must employ general categories of behavior as the criterion rather than specific consumption acts.

A NEW SCALING PROCEDURE FOR DISCRETE DATA

The Dillon and Madden paper is an extremely ambitious, provocative, and highly synthetic effort. They have integrated the principles of Guttman Scalograms with the analytic virtues of latent structure analysis while embodying the philosophical and practical attractiveness of building methods variation and measurement error into scale construction on discrete data.

I have selected in the following comments to raise some points regarding the general applicability and utility to consumer researchers of the Dillon and Madden model (DMM). (A rigorous mathematical evaluation is left to those who are more qualified to make such an assessment.)

An important first point is that the DMM is limited in its applicability to nominal data. This is so because the generally accepted procedures of Joreskog and Sorbom (1981) are, in LISREL V, applicable (perhaps arguably so) -co ordinal data and are capable of handling (in a confirmatory factor analysis mode) the same fundamental objective to which the Dillon and Madden effort is directed. However, it should be noted that the Dillon and Madden procedure provides specific statistical tests, whereas this IS infeasible when applying Joreskog and Sorbom's unweighted least squares (ULS) to discrete data.

A second issue concerns the domain of constructs to which the DMM is applicable. The Guttman Scalogram is the pivotal conceptual and analytical framework from which the Dillon and Madden model extends. Two assumptions underlie the Guttman scheme: unidimensionality of the scaled construct and cumulativeness of the items constituting the scale. This second assumption follows from the fact that attitude items in a Guttman scale have a strict logical ordering such that agreement with one implies a necessary agreement with another, it in turn with another, and so forth. Based on this logic, a limited number (k+1) of "correct" responses (or pure scale types) are logically permissible from the k items constituting the scale.

Although this is appropriate logic in the case of attitude scaling, its general applicability is questionable, and, to the extent the assumption doesn't hold, the Dillon and Madden procedure represents little more than all interesting exercise.

Consider, for example, a situation where a "search intensity" construct has been measured with a series of yes/no questions worded something such as: Prior to making your choice, did you inspect Consumer Reports, shop at more than three stores, talk with friends, and so forth. The data in this case are dichotomous and measurement error is probable, thus making the DMM potentially applicable in scaling "search intensity"; however, in the absence of strong rationale for why a "yes" response to one item would logically necessitate "yes" to another, the assumption of cumulativeness and the associated assumption of a restricted number of pure scale types would be inappropriate, thus limiting the applicability of the DE.

Dillon and Madden are to be complimented for this ambitious undertaking, which personifies the rapid advancements now taking place in the measurement of consumer behavior phenomena. The jury is still out, and the reader is advised to request the unabridged version or the Dillon and Madden paper (noted in their first footnote) in order to stake a complete assessment and to fully appreciate the development of their thinking.

REFERENCES

Ajzen, Icek and Fishbein, Martin (1980), Understanding Attitudes and Predicting Social Behavior, Englewood-Cliffs. NJ: Prentice-Hall.

Bagozzi, Richard P. (1980), Causal Models in Marketing, New York: John Wiles & Sons.

Brunswik, E. (1956), Perception and the Representative Design of Experiments, Berkeley, CA: University of California Press.

Buncher, Martin M. (1982), "Focus Groups Seem Easy to Do and Use, But They're Easier to Misuse and Abuse," Marketing News, (September 17), pp. 14-15, section 2.

Calder, Bobby J. (1977), "Focus Groups and the Nature of Qualitative Marketing Research," Journal of Marketing Research, 14, 353-64.

Campbell, Donald T. and Fiske, D.W. (1959), "Convergence and Discriminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin, 56, 81-105.

Cohen, Joel B. (19 g , "An Interpersonal Orientation to the Study of Consumer Behavior," Journal of Marketing Research, 4, 270-78.

Cox, Keith K., Higginbotham, James B., and Burton, John (1976), "Applications of Focus Group Interviews in Marketing," Journal of Marketing, 40, 77-80.

Cronbach, Lee J. (1975), "Beyond the Two Disciplines of Scientific Psychology," American Psychologist, 30, 116-27.

Fern, Edward F. (1982), "The Use of Focus Groups for Idea Generation: The Effects of Group Size, Acquaintanceship, and Moderator on Response Quantity and Quality," Journal or .Marketing Research, 19, 1-14.

Heeler, Roger M. and Ray, Michael L. (1972), "Measure VaLidation in Marketing," Journal of Marketing Research, 9. 361-70.

Joreskog, Karl G. and Sorbom, Dag (1981), LISREL V: Analysis of Linear Structural Relationships by Maximum Likelihood and Least Squares Methods, Chicago: National Educational Resources, Inc.

Kassarjian, Harold J. (1971), "Personality and Consumer Behavior- A Review," Journal of Marketing Research, 8, 409-19.

Kernan, Jerome B. (1977), "Social Class and Spending Behavior: Retrospective Comment," in Louis E. Boone (ed.), Classics in Consumer Behavior, Tulsa: Petroleum Publishing Company, 316-17.

Noerager, Jon B. (1979), "An Assessment of CAD--A Personality Instrument Developed Specifically for Marketing Research," Journal of Marketing Research, 16, 53-9.

Stern, G.G. (1970), People in Context, New York: John Wiley & Sons.

Woodside, Arch G. and Andress, Ruth B. (1976), "CAD Eight Years Later," Journal of the Academy of Marketing Science, 3

----------------------------------------